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As  the  U.S.  begins  a  major  build-up  and  re-equipment  of  its  Armed  Forces,  a  major 
challenge  facing  both  the  Services  and  industry  is  in  the  area  of  training,  a 
fact  clearly  reflected  in  these  Proceedings  of  the  Third  Interservice/Induotry 
Training  Equipment  Conference  held  in  Or lander,  Florida,  November  30  -  December  2. 


As  you  read  or  refer  to  this  record  of  the  conference,  there  are  several  salient 
^thoughts  which  I  would  like  to  share  with  you. 

First,  these  Proceedings  and  the  rest  of  the  conference  program  demonstrate  the 
extensive  efforts  to  which  both  industry  and  the  Services  have  gone  to  in  order 
to  make  this  annual  event  the  premier  conference  in  the  world  on  Training  and 
Simulation  Equipment. 

The  need  for  realistic  and  dependable  methods  for  training  operators  of  the  new 
weapon  systems  now  being  acquired  by  the  Armed  Services  is  increasing,  providing 
new  management  and  technical  challenges  to  industry.  Despite  advances  in  auto¬ 
mation  of  prime  equipment,  and  the  increased  use  of  built-in  training  capabilities 
in  that  prime  equipment,  the  Department  of  Defense  and  the  Armed  Services  continue 
to  have  significant  and  extensive  training  need3.  And  these  requirements  are 
clearly  going  to  continue  to  grow  throughout  the  198^' s. 
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Industry  response  to  these  needs,  as  reflected  in  these  Proceedings  and  in  the 
more  than  50  exhibits  at  the  conference,  has  been  truly  innovative.  Not  only 
is  industry  meeting  the  technical  and  management  challenges,  it  is  again  demon¬ 
strating  the  cost  effectiveness  of  training  equipment  over  the  use  of  operational 
equipment  to  solve  training  needs. 

In  sum,  the  edge  we  have  on  our  adversaries  is  the  of  high  technology  in  our 
weapons  systems.  Only  with  proper lyyirained  personnel  can  the  full  benefits  of 
that  technology  be  utilized.  We  are  Responding  to  that  challenge. 
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Robert  W,  Beck 

Aeronautical  Systems  Division 
U.S.  Air  Foret  Systems  Command 
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ABSTRACT 

The  performence  requirements  for  visual  systems  to  support  Air  Force  flight 
simulation  far  exceed  the  requirements  for  visual  systems  which  support  commercial 
airline  simulators.  The  difference  In  requirements  stems  from  the  diversity  and 
complexity  of  military  flight  missions.  While  visual  systems  for  airline  simulators 
are  generally  easily  defined  and  delivered  on  a  predictable  schedule!  most  Air  Force 
visual  system  procurements  are  not  so  straightforward.  The  difficulty  In  obtaining 
visual  systems  to  meet  Air  Force  requirements  In  a  predictable  manner  arises  largely 
from  the  fact  that  current  conmerclally  available  visual  systems  have  been  designed 
to  support  airline  type  missions;  The  expansion  of  these  systems  to  meet  Air  Force 
training  requirements  has  been  less  than  graceful.  This  paper  examines  the  differ¬ 
ences  between  airlines  and  Air  Force  flight  missions,  the  Impact  of  these  differences 
on  system  performance  requirements  and  the  resultant  challenges  In  Air  Force  visual 
system  procurement  for  both  the  Air  Force  and  the  contractor. 
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INTRODUCTION 

The  acquisition  of  visual  systems  for  com¬ 
mercial  airline  flight  simulators  (FAA  Phase  III 
requirements  notwithstanding)  1$,  relatively 
speaking,  a  straightforward  process.  A  visual 
system  supporting  airline  flight  training  must 
provide  training  for  taxi,  take-off,  and  landing. 
The  basic  cues  required  to  train  these  tasks  are 
relatively  well  defined  since  the  training  takes 
place  within  a  definable  environment— an  airport 
area.  Accordingly,  the  data  base  or  gaming  area 
requirements  are  small— on  the  order  of  one  hun¬ 
dred  square  miles  or  less.  The  data  base  contents 
can  usually  be  obtained  from  several  airport  blue¬ 
prints  and  city  street  maps.  The  visual  cues  for 
airline  tasks  normally  occur  In  the  pilot’s  for¬ 
ward  fleld-of-vlew  and,  at  most,  a  fleld-of-vlew 
of  plus  and  minus  90  degrees  horizontal  Is  re¬ 
quired,  A  narrower  fleld-of-vlew  Is  usually  suf¬ 
ficient,  In  addition,  training  of  taxi,  take-off 
and  landing  does  not  Involve  the  need  to  recognize 
or  track  relatively  small  features  at  long  ranges 
from  the  aircraft.  Hence,  only  a  moderate  system 
resolution,  on  the  order  of  6  to  10  arc  minutes, 

Is  required  In  order  to  provide  adequate  visual 
cues  for  these  tasks.  The  ability  to  define  the 
training  requirements  plus  the  fact  that  the  re¬ 
sultant  system  technical  requirements  are  all 
available  within  current  technology  has  resulted 
In  the  availability  of  numerous  "off-the-shelf" 
visual  systems  to  provide  high  fidelity  training 
for  the  commercial  airlines. 


For  the  most  part,  there  Is  no  such  thing  as 
"off-the-shelf"  when  It  comes  to  acquisition  of 
visual  systems  for  Air  Force  simulators.  The 
current  visual  system  product  lines  of  the  var¬ 
ious  simulator  equipment  manufacturers  are  all 
targeted  toward  end  tailored  for  the  commercial 
airline  market.  One  reason  for  this  Is  the  fact 
that  the  airline  training  scenario  Is  so  well  de¬ 
fined  and  stables  It  Is  easier  to  optimize  a  sys¬ 
tem  that  addresses  a  requirement  which  Is  not 
constantly  In  a  state  of  flux.  The  second  reason 
Is  two-fold  but  Is  clearly  a  matter  of  economics: 
1)  The  commercial  market  exists  for  a  standard 
visual  product  line  and  2)  Since  the  airline 
customer  can  specify  exactly  what  he  needs  In  a 


system  In  order  to  train  pilots,  the  vendor  will  ■ 
not  end  up  In  financial  jeopardy  trying  to  satisfy 
the  commercial  customer. 

THE  CHALLENGE 

The  challenge  In  acquiring  simulator  visual 
systems  for  the  Air  Force  stems  from  the  fact  that 
current  visual  system  technology  Is  oriented  to¬ 
ward  meeting  the  needs  of  the  commercial  airlines 
and  that  Air  Force  training  requirements  are  far 
more  diverse  and  demanding  In  terms  of  system  per¬ 
formance.  Although  take-off  and  landing  are  In¬ 
deed  key  flight  tasks  for  the  Air  Force  as  well  as 
the  airlines,  there  are  a  number  of  additional 
tasks  which  must  also  be  trained  In  the  simulator. 
These  tasks  are  as  follows: 

-Aerial  refueling 

-Air  combat 

-Formation  flight 

-Air  to  surface  weapons  delivery 

-Low  altitude  navigation  (terrain  avoidance/ 
terrain  following) 

With  few  exceptions,  these  tasks  cannot  be  satis¬ 
fied  by  a  visual  system  which  was  designed  primar¬ 
ily  to  train  take-off  and  landing.  Hence  the  chal 
lenge:  Provide  the  Air  Force  user  a  visual  system 
which  will  provide  not  only  take-off  and  landing 
training  but  also  training  In  various  tactical  and 
strategic  tasks  In  between.  It  will  be  shown  that 
this  challenge  belongs  to  both  the  Air  Force  and 
the  visual  system  contractors. 

Defining  the  Problem 

Each  of  the  training  tasks  listed  above  requires  a 
unique  set  of  visual  cues  to  be  displayed  to  the 
aircrew.  For  example,  aerial  refueling  requires 
mainly  a  well  defined  tanker  aircraft  to  be  dis¬ 
played  moving  In  six  degrees  of  freedom  relative 
to  the  ownship  and  little  or  no  terrain  surface 
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Information.  Low-level  navigation  requires  pri¬ 
marily  the  display  of  high  fidelity  terrain  and 
cultural  Information.  In  order  to  provide  the 
cues  for  a  given  task  In  a  particular  visual  sys¬ 
tem,  certain  critical  performance  characteristics 
are  required.  The  major  system  characteristics 
dictated  by  these  cue  requirements  can  be  con¬ 
densed  Into  the  following: 

1.  Large  fleld-of-vlew 

2.  High  resolution 

3.  Large  gaming  area 

4.  High  scene  content 

5.  Special  effects/techniques 

Figure  1  shows  how  each  of  these  character¬ 
istics  are  related  to  the  various  Air  Force  train¬ 
ing  tasks.  For  the  purposes  of  this  paper,  "large 
fleld-of-vlew"  Is  taken  to  mean  a  horizontal 
fleld-of-vlew  of  from  130  to  360  degrees  and  a 
vertical  fleld-of-vlew  of  from  36  to  360  degrees. 
"High  resolution"  Is  regarded  as  a  system  resolu¬ 
tion  In  excess  of  four  arc-minutes.  "High  scene 
content"  Implies  a  displayed  scene  density 
throughout  most  of  the  gaming  area  which  Is  great¬ 
er  thar  that  found  In  an  airfield  data  base.  The 
term  "large  gaming  area"  denotes  an  Instantaneous 
on-line  data  base  covering  multiple  thousands  of 


square  miles  In  area,  "Special  effects/techniques" 
Is  a  catch-all  reference  to  system  features  such  as 
the  depiction  of  artillery  shell  tracers,  muzzle 
flashes,  surface  to  air  missile  launches  and  so 
forth.  It  is  Important  to  note  that  none  of  these 
system  characteristics  are  found  In  standard,  off- 
the-shelf  commercial  visual  systems.  It  Is  also 
Important  to  note  that  In  full  mission  weapons  sys¬ 
tem  trainers  (WSTs)  the  requirement  exists  to  train 
several  of  the  discrete  tasks  listed  above  In  a 
single  simulator.  As  an  example,  a  typical  multi¬ 
role  fighter  aircraft  mission  scenario  Is  depicted 
In  Figure  2.  Grouping  the  various  mission  seg¬ 
ments  together,  It  can  be  seen  that  the  visual  sys¬ 
tem  for  a  WST  for  this  aircraft  must  be  capable  of 
providing  visual  cues  for  take-off,  aerial  refuel¬ 
ing,  low-level  navigation,  terrain  avoidance,  air 
to  surface  weapons  delivery,  air  to  air  combat  and 
landing.  Referring  back  to  Figure  1.  It  can  be 
seen  that  such  a  system  must  have  all  four  of  the 
critical  system  characteristics  discussed  earlier. 
Since  none  of  these  characteristics  are  available 
in  off-the-shelf  visual  systems,  the  result  Is  an 
acquisition  program  which  Is  largely  a  research 
and  development  effort. 

This  example  Is,  to  some  extent,  a  "worst- 
case"  situation  In  which  the  visual  system  must 
satisfy  a  wide  range  of  task  requirements  and, 
therefore,  Involves  all  of  the  major  critical  sys¬ 
tem  characteristics.  The  Air  Force  simulator  de¬ 
velopment  program  originally  aimed  at  developing  a 
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visual  system  to  meet  all  these  requirements  (Proj¬ 
ect  2360,  Tactical  Combat  Trainer)  was  clearly  the 
most  challenging  visual  system  development  program 
ever  Initiated  by  the  Air  Force.  The  basic  thrust 
of  Project  2360  was  to  develop  a  single  visual  sys¬ 
tem  which  would  provide  training  In  both  air  to  air 
combat  and  air  to  surface  weapons  delivery.  Prior 
to  the  program's  termination  due  to  funding  diffi¬ 
culties,  a  significant  amount  of  new  technology  In 
both  the  Image  display  and  Image  generation  areas 
was  developed.  The  challenge  still  remains  to  In¬ 
tegrate  this  new  technology  Into  a  complete  visual 
system. 

Development  Required 

The  concept  of  development-oriented  visual 
system  acquisition  programs  cannot  be  reserved  for 
large  scale  development  efforts  like  Project  2360, 

A  close  Inspection  of  Figure  1  will  reveal  the 
fact  that  only  three  of  the  training  tasks  do  not 
require  at  least  one  of  the  major  critical  system 
requirements.  Hence,  any  acquisition  program  for 
a  visual  system  which  Is  Intended  to  train  other 
than  take-off  and  landing,  Instrument/visual  tran¬ 
sition  and  air  refueling*  must  be  considered  a 
development  effort.  This  leads  to  the  conclusion 
that  "off-the-shelf"  procurement  of  visual  systems 
for  Air  Force  simulators  Is  limited  to  visual  sys¬ 
tems  for  medium  range  conventional  airlift  air¬ 
craft  simulators,  trainer  aircraft  simulators  and 
possibly  part-task  trainers  for  air  refueling. 

In  addition  to  the  availability  of  off-the- 
shelf  visual  systems,  the  commercial  airlines  also 
enjoy  another  advantage  not  enjoyed  by  the  Air 
Force.  All  of  their  aircraft  have  the  same  mis¬ 
sion-transport  passengers  and/or  cargo  from  point 
A  to  point  B.  The  Air  Force,  on  the  other  hand, 
has  some  aircraft  that  drop  bombs,  some  aircraft 


‘Recurrent  aerial  refueling  training  In  a  simu¬ 
lator  may  require  a  wide  horizontal  fleld-of-vlew, 
therefore  making  Its  acquisition  program  a  devel¬ 
opment  effort  as  well. 


» 


3 


that, •shoot  at  other  aircraft;  some  aircraft  that 
drop  .{Cargo  at  low  altitudes;  some  rotary  wing  air¬ 
craft  that  fly  below  tree-top  level  as  well  as 
aircraft  that  carry  passengers  and/or  cargo  from 
point  A  to  point  B.  The  point  to  be  made  here  Is 
that  virtually  every  different  aircraft  type  Im¬ 
plies  a  unique  set  of  simulator  visual  system  per¬ 
formance  requirements.  In  terms  of  the  acquisi¬ 
tion  process,  the  learning  curve  ends  up  being 
essentially  flat;  New  requirements  must  be  spec¬ 
ified  for  each  visual  system  acquisition  and  de¬ 
velopment  of  new  technology  or  Integration  of 
existing  technology  Into  a  new  application  must  be 
pursued. 


BOUNDING  THE  PROBLEM 
Estimating  Cost  and  Schedule 

In  terms  of  acquisition,  a  prime  advantage  of 
off-the-shelf  visual  systems  Is  cost  and  schedule 
predictability.  Commercial  visual  systems  are 
known  quantities;  Their  hardware  and  software  are 
largely  modular.  For  example,  a  wide  horizontal 
fleld-of-view  will  require  multiple  display  system 
modules  and  an  equal  number  of  additional  Image 
generation  channels.  The  cost  of  additional  air¬ 
field  data  bases  will  be  a  multiple  of  the  cost  of 
one  airfield  data  base  since  the  required  data  base 
content  for  any  given  airfield  Is  defined  or  read¬ 
ily  deflneable  and  obtainable.  Since  most  vendors 
of  commercial  visual  systems  have  established  price 
lists,  the  cost  of  hardware  and  software  (excluding 
unique  Integration  requirements)  can  be  readily  ob¬ 
tained.  Schedule  Is  tied  mainly  to  component  part 
lead  times  and  the  complexity  of  the  Integration 
task.  When,  as  In  the  case  of  the  Air  Force,  each 
new  visual  system  has  a  unique  set  of  requirements 
and  many  of  those  requirements  push  or  clearly  ex¬ 
ceed  the  current  technical  state-of-the-art,  ar¬ 
riving  at  what  the  cost  and  schedule  should  be  for 
a  given  visual  system  requires  a  great  deal  of 
creative  estimating  skill  and  a  few  outright 
guesses.  Figures  3  and  4  are  an  attempt  to  demon¬ 
strate  the  difficulty  In  defining  how  much  a  par¬ 
ticular  non-commercial  visual  system  should  cost. 


Fleld-of-Vlew  Figure  3  depicts  the  problem 
In  estimating  the  cost  of  a  visual  display  syster 
In  terms  of  fleld-of-vlew.  The  commercial  hard¬ 
ware  values  are  predicated  on  using  standard  CRT/ 
mirror/beamsplitter  type  display  modules.  Dis¬ 
regarding  the  cost  of  the  ancillary  mounting  hard¬ 
ware  and  keeping  resolution  a  constant,  the  cost 
of  the  commercial  hardware  Is  simply  the  desired 
fleld-ofivlew  In  number  of  displays  (units  of 
about  44°)  horizontally  multiplied  by  the  desired 
fleld-of-vlew  In  number  of  displays  (units  of 
about  32°)  vertically  (maximum  vertical  multiplier 
Is  2  for  a  total  of  64°),  Once  the  horizontal 
fleld-of-vlew  reaches  about  200°  and  the  vertical 
fleld-of-vlew  reaches  about  64°  the  ability  to 
readily  estimate  display  cost  diminishes  rapidly, 
"Standard"  CRT/mlrror/beamspl Itter  type  displays 
can  no  longer  be  used  and  hence  the  solid,  known 
cost  estimating  base  disappears.  This  Is  not  to 
say  that  a  system  cannot  be  costed,  but  the  prob- 
letn  does  Indeed  become  mu  'tlvarlatc,  Once  the 
bounds  of  modular  display  systems  are  exceeded, 
horizontal  fleld-of-vlew  can  be  traded  off  against 
vertical  fleld-of-vlew  and  vice  versa.  Resolution 
also  becomes  a  tradeoff  against  fleld-of-vlew, 

While  there  are  available  cost  figures  for  a  few 
large  fleld-of-vlew  displays,  each  display  repre¬ 
sents  a  unique  level  of  performance  and  successful 
Interpolation  and  extrapolation  of  those  cost  fig¬ 
ures  to  displays  with  different  characteristics  Is 
Impossible.  Given  a  set  of  state-of-the-art  dis¬ 
play  components  (e.g.,  projectors,  screens,  etc.) 
and  a  particular  set  of  performance  requirements. 

It  Is  a  relatively  straightforward  process  to  de¬ 
termine  the  number  of  components  required  and  hence 
produce  a  fairly  reliable  cost  estimate.  Changing 
any  one  variable,  l.e,,  horizontal  or  vertical 
fleld-of-vlew  ,  projector  type  or  resolution  will 
require  a  new  estimate. 

If  one  attempts  to  meet  a  given  set  of  per¬ 
formance  requirements  using  display  components 
which  are  not  proven,  state-of-the-art  devices, 
cost  and  schedule  risks  Increase  significantly. 

The  completion  date  for  the  display  subsystem  be¬ 
comes  Intimately  tied  to  the  availability  dates  of 
the  various  R  &  D  components.  The  cost  of  the 
display  subsystem  Is  at  the  mercy  of  the  final 
production  cost  of  these  components  as  well. 

Resolution.  Attempting  to  cost  out  a  high 
resolution  visual  system  Is  very  much  akin  to  the 
large  fleld-of-vlew  problem.  Per  the  previous 
discussion  on  fleld-of-vlew,  once  the  fleld-of- 
vlew  capability  of  juxtaposed  standard  display 
units  Is  exceeded,  resolution  becomes  a  dependent 
variable  In  the  resolutlon/fleld-of-vlew  equation. 
Figure  4  demonstrates  this  concept.  For  standard 
display  modules,  Increasing  display  resolution  can 
be  thought  of  as  a  linearly  Increasing  variable. 

For  a  given  resolution,  the  variable  becomes  a  con¬ 
stant  regardless  of  the  chosen  total  fleld-of-vlew. 
For  flelds-of-vlew  that  exceed  the  capability  of 
the  modular  display  approach,  arriving  at  a  system 
cost  requires  a  great  deal  more  analysis.  Oue  to 
the  Inverse  relationship  between  field-of-vlew  and 
resolution  and  the  resultant  available  design 
trade-offs,  the  question  of  the  cost  of  higher  res¬ 
olution  must  always  take  Into  account  the  fleld-of- 
vlew,  This  discussion  has  conveniently  Ignored  the 
Impact  of  Image  generation  on  system  resolution. 
However,  the  relationship  between  resolution  and 
fleld-of-vlew  follows  essentially  the  same  rules 
as  the  displays. 


Gaming  Area  (Data  Base).  As  mentioned  earli¬ 
er,  the  gaming  areas  or  data  bases  for  airline  sim¬ 
ulators  are  oriented  to  the  airfield  Itself  and  a 
very  small  geographical  area  surrounding  It.  The 
required  contents  of  the  data  base  are  fairly  well 
understood  by  the  contractor  and  hence  a  "cookbook" 
pricing  scheme  can  be  adopted  for  pricing  one  or 
more  airfield  data  bases.  The  work  required  to 
create  each  different  airfield  data  base  Is,  plus 
or  minus  ten  percent  for  unique  significant  fea¬ 
tures,  the  same.  Also,  since  the  quantity  of  work 
required  to  create  a  data  base  Is  fairly  well 
known,  the  amount  of  time  It  takes  to  create  the 
data  base  Is  largely  a  function  of  the  manpower 
applied  to  the  task.  Schedule  confidence  Is  en¬ 
hanced  by  the  fact  that  the  techniques  used  to 
build  small  area  airfield  oriented  data  bases  (eg., 
digitizing  from  maps)  are  completely  understood. 

The  cost  and  schedule  requirements  for  creat¬ 
ing  large  data  bases  for  Air  Force  systems,  cannot 
be  arrived  at  with  anywhere  near  equal  ease.  There 
are  several  reasons  for  this:  1)  First  and  fore¬ 
most,  the  data  base  content  reoulred  for  any  given 
mission  task  (l.e.,  low-level  navigation,  tactical 
air  to  surface  weapons  delivery,  etc.)  Is  not  well 
understood.  For  the  take-off  landing  task  we  know 
that  most  of  the  visual  cues  occur  on  or  In  direct 
proximity  to  the  runway.  Scene  complexity  (except 
for  textural  Information  In  the  area  Immediately 
adjacent  to  the  landing  zone)  Is  not  a  significant 
factor  In  building  an  airfield  data  base.  The 
fidelity  of  the  representation  of  the  runway  envi¬ 
ronment  Is  known,  by  experience,  to  be  paramount. 
With  little,  if  any,  experience  with  the  creation 
and  training  use  of  data  bases  for  low-level  navi¬ 
gation,  for  Instance,  it  Is  difficult  to  estimate 
the  per- square-mile  manpower  required  to  build  the 
data  bare.  2)  Availability  of  source  data  also 
places  a  question  mark  on  the  creation  of  large 
data  bases.  For  a  given  airfield,  civil  engineer¬ 
ing  blueprints  are  normally  available  and  augment¬ 
ing  that  data  with  photographs  Is  a  manageable 
task.  If  one  considers  the  problem  of  creating  a 
data  base  which  represents  a  large  area  of  the  real 
world,  the  acquisition  of  source  data  becomes  a 
tenuous  proposition. (1 )  Given  that  a  large  part  of 
the  continental  U.S.  Is  now  available  In  digital 
format,  the  current  cultural  Information  In  that 
data  Is  not  sufficient  to  do  a  credible  job  of 
visual  data  base  generation.  There  are  numerous 
characteristics  of  the  visual  world  which  are  not 
currently  portrayed  In  the  data,  not  the  least  of 
which  are  coio, ,  roads,  railroads,  etc.  Additional 
features  like  roads  must  be  digitized  from  charts. 
Color  Information  must  either  be  Inferred  from 
feature  descriptors  or  a  sampling  of  photography. 

3)  Conventional  techniques  used  In  creating  small 
area  data  bases,  such  as  hand  digitizing  of  fea¬ 
tures,  are  Impractical  when  producing  a  data  base 
covering  multiple  thousands  of  square  nautical 
miles.  If,  for  example,  It  takes  five  data  base 
modellers  a  total  of  six  months  to  create  a  single 
airfield  area  covering  50  square  miles,  simple 
arithmetic  reveals  that  It  would  take  2,500  man- 
years  to  create  a  50,000  square  mile  data  base 
using  the  same  technique,**  The  obvious  answer 
Is  some  type  of  automated  data  base  generation 


♦♦Creation  includes  collection  of  source  data, 
digitization  and  debug  on  the  real-time  visual 
system. 
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system  employing  a  computer  transformation  of  the 
digital  data  base.  This,  however,  requires  the 
development  of  additional  software  to  perform  this 
work.  The  estimation  of  the  amount  of  software 
needed  to  do  the  job  Is  difficult  because  the  end 
product  (the  on-line  data  base)  must  be  compatible 
with  the  host  Image  generator.  The  Image  genera¬ 
tor  Is  normally  In  the  Initial  stages  of  design 
when  the  data  base  software  needs  to  be  developed, 
thereby  making  this  estimation  task  more  difficult. 

Scene  Content.  Perhaps  the  most  overlooked 
area  In  the  discussion  of  visual  system  perfor¬ 
mance  Is  that  of  scene  content--What  visual  Infor¬ 
mation  needs  to  be  displayed  to  the  aircrew?  What 
does  the  aircrew  need  to  see  on  the  visual  dis¬ 
plays  In  order  to  receive  some  measure  of  train¬ 
ing?  The  answers  to  the  questions  are  different 
for  each  of  the  different  tasks  that  need  to  be 
trained,  and  hence  virtually  every  different  visu¬ 
al  system  procured  by  the  Air  Force.  The  diver¬ 
sity  of  the  various  tasks  and  the  quest  for  the 
answers  to  the  above  questions  represent  a  signif¬ 
icant  challenge  In  the  Air  Force's  acquisition  of 
visual  systems. 

The  solution  to  the  scene  content  problem  Is 
shared  between  two  system  design  areas:  1)  The 
data  base  content  and  2)  The  Image  generator 
power.  Both  of  these  areas  are  Intimately  re¬ 
lated  to  each  other.  Too  little  data  base  content 
will  result  in  Insufficient  Image  density  (clut¬ 
ter)  to  provide  suitable  training.  Too  much  data 
base  content  can  overload  the  Image  generator  and 
product  Image  anomalies  which  result  In  reduced 
training  effectiveness. 

In  the  final  analysis,  it  is  largely  by  the 
overall  scene  content  that  the  "goodness"  or  ade¬ 
quacy  of  a  visual  system  Is  evaluated.  If  an  air¬ 
crew  attempts  to  train  In  a  simulator  with  a  visu¬ 
al  system  which  does  not  provide  cues  which  are 
analogous  to  actual  aircraft  flight  experience, 
the  visual  system  will  be  judged  as  unacceptable 
or  unuseable  for  training.  One  arc -minute  reso¬ 
lution  and  a  360°  field-of-view  will  not  compensate 
for  Incoherent  scene  content.  It  Is  in  this  area 
that  the  visual  system  contractor  gets  an  oppor¬ 
tunity  to  share  the  visual  system  acquisition 
challenge  with  the  Air  Force. 


visual  system  program,  the  Inclusion  of  special 
effects  can  have  a  significant  impact  on  the  Image 
generator  hardware  and  the  development  schedule. 

In  many  cases,  an  Image  generator  must  be  designed 
from  the  ground  up  In  order  to  accommodate  special 
effects.  In  some  cases,  the  system  designer  may 
believe  that  a  particular  special  feature  can  be 
produced  In  a  system  by  simply  modifying  or  aug¬ 
menting  an  existing  standard  feature  (e.g.,  using 
the  landing  light  subsystem  to  do  simulation  of 
flare  Illumination  effects).  Unfortunately,  It 
usually  Isn't  until  the  hardware  Is  designed, 
built  and  operating  before  It  is  determined  that 
the  simple  modification  approach  will  not  satisfy 
the  special  effect  requirements.  The  result  Is 
usually  a  large  hardware  redesign  effort,  hardware 
and  software  growth,  along  with  the  attendant 
schedule  impacts. 

MEETING  THE  CHALLENGES 

At  this  point.  It  should  be  clear  that  acquir¬ 
ing  visual  systems  which  meet  Air  Force  training 
requirements  Is  anything  but  a  straightforward, 
"place  your  order"  process.  The  challenges  are 
significant.  It  should  also  be  clear  that  the 
challenges  are  shared  by  the  Air  Force  and  the  con¬ 
tractor.  Achieving  success--produc1ng  a  usable 
visual  system  for  the  user--MAC,  SAC,  TAC,  or  ATC, 
requires  Initiatives  on  the  part  of  both  the  Air 
Force  and  the  contractor.  Some  of  these  Initia¬ 
tives  may  require  a  departure  from  the  way  things 
are  usually  done.  Here  are  some  places  to  start: 

Fire  The  Artist 

Step  one  In  bringing  order  to  the  chaos  of 
visual  system  development  belongs  to  the  contractor. 
A  picture  may  be  worth  a  thousand  words,  but  the 
words  often  end  up  being  lies  when  the  final  system 
doesn't  produce  scenes  anywhere  near  the  8  x  10 
"artist  concept"  photographs.  Describing  the  per¬ 
formance  of  a  system  which  doesn't  even  exist  on 
paper  by  means  of  a  colorful  picture  Is  a  heinous 
act.  The  user  will  expect  the  system  output  to 
look  just  like  the  pictures.  Why  shouldn't  he? 

When  It  doesn't,  he'll  think  you've  deceived  him 
and  that  you  can't  be  trusted.  Why  shouldn't  he? 
Regaining  the  user's  confidence  and  convincing  him 
that  he  likes  your  system  will  be  most  difficult. 


Achieving  an  optimal  scene  content  can  make  or 
break  an  entire  visual  system  program  In  terms  of 
cost  and  schedule  as  well,  Making  the  assumption 
that  the  Image  generator  can  generate  sufficient 
scene  density  (total  number  of  faces,  edges,  light 
points,  etc.)  the  only  scene  density  variable  Is 
the  data  base  content.  The  contractor.  In  concert 
with  the  eventual  users  (aircrews),  must  begin 
early  on  in  a  program  to  try  and  determine  the 
optimal,  useable  scene  content.  Failure  to  do  an 
adequate  job  of  data  base  definition  on  the  front- 
end  runs  the  risk  of  generating  an  entire  multi¬ 
thousand  mile  data  base  with  Incorrect  or  improp¬ 
erly  portrayed  features.  The  cost  and  schedule  Im¬ 
pact  of  regenerating  an  entire  large  area  data  base 
is  staggering. 

Special  Effects/Techniques.  Since  most  visual 
systems  are  designed  to  satisfy  commercial  training 
requirements,  the  special  effects  required  for  some 
A1,  Force  training  tasks  (l.e.  depiction  of  flares, 
tracers,  weapons  Impacts,  etc.)  present  challenges 
In  Image  generator  design.  In  the  course  of  the 


Understand  The  Requirements 


This  Initiative  belongs  to  both  the  Air  Force 
program  team  and  the  contractor.  It  Is  Impossible, 
solely  by  means  of  a  technical  performance  specifi¬ 
cation,  to  convey  to  the  contractor  all  of  the  In¬ 
formation  regarding  what  the  user  needs  to  see  In 
the  visual  system  in  order  to  train.  The  Air  Force 
needs  to  do  a  better  job  of  describing  the  various 
training  tasks  to  the  contractor.  The  technical 
specification  ends  up  being  the  objective  criteria 
for  visual  system  acceptance,  but  the  subjective 
criteria  (scene  content,  lack  of  distracting  arti¬ 
facts,  etc.)  are  equally  Important  in  achieving  a 
successful  program.  The  contractor  needs  to  Inves¬ 
tigate  the  actual  training  tasks  long  before  system 
design  begins;  The  proposal  preparation  period  Is 
not  too  early  to  begin. 


Gaining  the  initial  understanding  Is  Important, 
but  the  task  continues  throughout  the  life  of  the 
program.  Continuous  interaction  between  the  de¬ 
signers  and  the  eventual  users  Is  essential.  As 


discussed  earlier,  one  set  of  bad  assumptions  made 
early-on  In  a  program  can  spell  disaster.  Getting 
the  user's  subjective  comments  as  soon  as  they  can 
see  something  displayed  on  the  system  can  help  to 
avert  problems  down  the  line  on  both  sides.  The 
Air  Force  must  make  a  serious  effort  to  try  and 
ensure  that  the  same  people  who  evaluate  and  cri¬ 
tique  the  system  this  month  will  be  available  next 
month  as  well.  Failure  to  do  thl:  will  result  in 
conflicting  comments  and  frustration  on  the  part 
of  the  contractor's  personnel. 

Systems  Engineering 

The  requirement  for  "systems  engineering" 
appears  In  every  Air  Force  solicitation  for  simu¬ 
lators.  Contractor  proposals  always  contain  de¬ 
tailed  descriptions  of  how  systems  engineering  will 
be  pursued  In  the  candidate  program.  If  there  Is 
one  area  of  simulation  that  demands  the  application 
of  a  systems  oriented  design  approach.  It  Is  surely 
visual  simulation.  But  for  some  reason,  visual 
systems  traditionally  suffer  from  a  gross  lack  of 
systems  engineering  discipline. 

There  appear  to  be  several  reasons  why  this  Is 
true.  One  reason  Is  certainly  the  fact  that,  the 
technology  of  visual  simulation  Is  growing  so  fast 
that  subsystem  designs  never  stabilize  long  enough 
to  be  totally  quantified.  This  phenomenon  Is  ag¬ 
gravated  by  the  fact  that  military  simulation  re¬ 
quirements  always  seem  to. press  the  state-of-the- 
art  to  Its  limits.  When  the  characteristics  of  a 
given  subsystem  are  not  well  understood,  the  Impact 
and  Interaction  of  that  subsystem  with  the  rest  of 
the  system  and  the  resultant  requirements  It  places 
on  the  design  of  adjacent  subsystems  cannot  be 
understood  either.  The  result  Is  a  system  composed 
of  subsystems  which  may  not  function  together  as  a 
unit  or,  at  best,  operate  well  below  system  speci¬ 
fication  requirements. 

The  second  reason  for  marginal  systems  engi¬ 
neering  stems  from  the  attempt  to  force  existing 
designs  to  do  things  they  were  never  Intended  to 
do.  Rather  than  starting  from  the  ground  up  In 
systems  design,  a  decision  Is  made,  before  the  re¬ 
quirements  are  fully  understood,  to  use  a  specific 


existing  subsystem  design.  The  number  of  successes 
using  this  approach  are  few.  While  the  decision  to 
use  this  approach  Is  based  on  a  motive  of  fiont-end 
cost  avoidance,  the  long  term  result  Is  often  one 
of  false  economy.  It  often  costs  more  to  redesign 
a  system  once  It  Is  built  than  It  would  have  to  do 
the  job  carefully  and  correctly  from  the  start. 

Systems  engineering  Is  not  merely  ensuring 
that  the  various  subsystems  wTTl  fit  together,  or 
Interface  properly.  The  various  components  must, 
when  brought  together,  form  a  totally  Integrated 
system.  In  the  world  of  visual  systems,  tnls  means 
that  the  data  base  people,  the  Image  generation 
people  and  the  displays  people  all  fully  understand 
each  other's  design  objectives.  The  data  base  de¬ 
signers  may  create  a  beautiful  set  of  data  base 
files,  but  If  those  files  contain  too  much  data  for 
the  Image  generator  to  process  In  a  given  frame 
time  when  the  two  are  Integrated,  the  choice  Is 
either  to  regenerate  the  data  base  or  redesign  the 
Image  generator.  Either  option  Is  costly. 

CONCLUSIONS 

The  business  of  acquiring  visual  systems  for 
Air  Force  simulators  has  been  shown  to  be  dramati¬ 
cally  different  from  the  same  task  for  the  commer¬ 
cial  airlines.  Accordingly,  the  acquisition  pro¬ 
cess  must  be  pursued  differently  by  both  the  Air 
Force  and  the  contractors.  The  key  to  success  on 
the  part  of  both  parties  Is  continuous,  open  Inter¬ 
action  throughout  the  life  of  the  program.  The 
Air  Force  needs  to  do  a  better  job  of  conveying  to 
the  contractor  exactly  what  the  training  require¬ 
ments  are.  But  the  Air  Force  must  continue  to  rely 
on  the  contractor  conmunlty  to  dllllgently  and  1n- 
novatlvely  convert  those  training  requirements  Into 
the  Integrated  visual  system  that  will  do  the  job 
required. 
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ABSTRACT 

A  visual  simulation  system  design  Is  described  which  provides  an 
observer  seated  In  a  cockpit  with  an  apparent  high  resolution  display 
over  a  wide  field  of  view  limited  only  by  cockpit  structure.  The  system 
utilizes  a  Helmet  Mounted  Opto-mechanical  Lase*1  Projector  to  produce  a 
composite  display  on  a  high  gain  screen  surrounding  the  cockpit.  The 
display  consists  of  two  full  color  laser  rasters  comprising  an  Inset 
and  a  surround.  One  raster  Is  dedicated  to  a  relatively  narrow,  high 
resolution  area  of  Interest  which  tracks  the  observer's  look  direction. 
The  other  raster  provides  a  wide,  low  resolution  Instantaneous  field  of 
view  in  the  surrounding  area  corresponding  to  the  observer's  peripheral 
field.  The  other  major  system  components  are  a  head  attitude  sensor, 
an  eye  attitude  sensor  and  a  two  channel  computer  Image  generation 
system  whose  performance  Is  tailored  to  the  display  requirements. 


INTRODUCTION 

Simulators  are  utilized  In  military  flight 
training  to  provide  the  pilot  or  other  aircrew 
with  an  interactive  environment  within  which  he  can 
learn  and  exercise  the  skills  required  to  operate 
his  weapon  system.  Tasks  such  as  low  altitude 
flight,  navigation,  target  acquisition  and  weapon 
delivery,  threat  avoidance,  and  confined  area 
maneuvering  are  performed  in  a  large  complex,  dyna¬ 
mic  visual  environment.  The  cost/training  effec¬ 
tive  simulation  of  such  an  environment  In  a  ground 
based  training  system  Is  a  goal  of  visual  simula¬ 
tion  technology. 

The  historic  approach  to  providing  a  wide  field 
of  view,  high  resolution  display  has  been  to 
mosaic  a  large  number  of  display  windows  around 
the  trainee.  The  number  of  display  windows  or 
channels  required  for  such  an  approach  is  a 
function  of  the  size  of  the  desired  field  of  view, 
the  desired  resolution,  and  the  number  of  picture 
elements  (pixels)  which  can  be  provided  by  a  window. 
The  state-of-the-art  for  typical  display  window 
capability  is  approximately  one  million  pixels. 

A  field  of  view  requirement  of  two  thirds  of  a 
complete  sphere  combined  with  a  resolution  require¬ 
ment  for  pixels  to  subtend  two  arc  minutes  Implies 
more  than  thirty  display  windows.  The  image 
generator  for  such  a  display  system  would  also 
have  thirty  channels.  It  Is  obvious  that  the 
mosaic  approach  becomes  more  and  more  Impractical 
as  the  field  of  view  increases  and  the  desired 
resolution  improves.  But  what  are  the  alternatives? 

An  alternative  approach  is  to  take  advantage  of 
the  perceptual  limitations  of  the  observer.  The 
observer  does  not  see  the  entire  available  field 
at  any  instant  in  time.  His  instantaneous  field 
of  view  is  a  fraction,  albeit  a  large  fraction,  of 
the  total  field  available  to  him  through  head  and 
body  movements.  Nor  does  the  observer  see  his 
entire  Instantaneous  field  at  high  resolution.  His 
high  resolution  seeing  is  confined  to  a  relatively 
small  area  of  interest  surrounding  his  look 
direction.  The  Helmet  Mounted  Laser  Projector 
visual  simulation  system  is  designed  to  provide  a 
display  which  efficiently  matches  the  observer's 
capabilities. 


The  basic  system  concept  has  been  reported 
(1).  However,  it  will  be  briefly  summarized  so 
that  the  analyses  and  experimental  results  reported 
in  this  paper  can  be  understood  in  proper  context. 

SYSTEM  CONCEPT 

The  visual  simulation  concept  is  based  on  the 
premise  that  a  composite  display  consisting  of  an 
eye  tracked  area  of  interest  (AOI)  surrounded  by  a 
head  directed  instantaneous  field  of  view  (IFOV) 
would  be  perceived  by  the  observer  as  having  high 
resolution  throughout  his  available  field  of  view. 

The  technical  approach  chosen  to  implement 
this  concept  is  the  Helmet  Mounted  Laser  Projector 
Visual  Simulation  System.  The  AOI  and  the  IFOV  are 
each  produced  by  a  full  color  laser  raster.  The 
composite  display  is  projected  from  the  observer's 
helmet  through  a  single  projection  lens  onto  a 
retroreflective  spherical  screen.  The  lasers, 
modulators:  and  line  scanner  are  located  remote 
from  the  observer.  The  modulated  laser  lines  are 
relayed  to  the  observer's  helmet  by  a  flexible, 
lightweight  fiber  optic  link.  The  helmet  mounted 
optical  system  performs  several  functions;  it 
converts  the  two  line  scans  into  two  rasters, 
combines  the  two  rasters  into  a  single  composite 
frame,  offsets  the  composite  frame  to  follow  eye 
movements  and  to  compensate  for  computational  lag 
in  the  computer  image  generator  (CIG),  and  projects 
the  composite  display  onto  the  screen.  Figure  1 
shows  a  schematic  diagram  of  the  display  optical 
system.  The  remaining  major  components  of  the 
visual  simulation  system  include  a  head  attitude 
sensor,  an  eye  attitude  sensor,  and  a  CIG. 

The  reasons  for  choosing  this  technical 
approach  are  discussed  in  a  previous  paper  (1)  and 
will  not  be  repeated  here  .  They  may  be  summarized 
by  stating  that  the  potential  advantages  outweighed 
the  risks. 

A  summary  of  the  Helmet  Mounted  Laser  Projector 
Visual  Simulation  System  performance  goals  is 
given  in  Table  I . 
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TABLE  I  SYSTEM  PERFORMANCE  GOALS 


Apparent  Field  of  View  - 

Apparent  Resolution  - 

Displayed  Instantaneous  Field  of  View  - 

Displayed  Area  of  Interest  - 

AOI  Resolution  - 

IFOV  Resolution  - 

Apparent  Luminance  - 

Color  - 

Contrast  Ratio  - 

An  artist's  concept  of  the  system  is  in  Figure 
2.  Note  that  the  view  Is  that  of  scmeone  looking 
over  the  observer's  shoulder.  The  observer,  him¬ 
self,  would  not  be  aware  of  the  composite  nature 
of  the  display.  Nor  would  he  be  aware  of  the 
absence  of  display  outside  his  Instantaneous 
field  of  view. 


Limited  Only  by  Cockpit  Structure 
1.7  Arc  Minute/Pixel 
145°  Diagonal 
36°  Diagonal 

1.7  Arc  Minute/Pixel  on  -  Axis 
6.5  Arc  Minute/Pixel  on  -  Axis 
10  Foot-Lamberts  (Highlight) 

Full 

30:1 

smoothly  varying  transition  region  combined  with  a 
delay  of  80  milliseconds  and  an  eye  tracker 
accuracy  of  +  2.50  would  cause  noticeable,  but  not 
objectionable,  perception  of  the  borders  of  the 
AOI. 

Instantaneous  Field  of  view 
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VISION  MODELS 

Before  proceeding  with  the  design  and  fabrica¬ 
tion  of  the  visual  simulation  system  based  on  the 
helmet  mounted  laser  projector,  several  questions 
needed  answers.  How  large  should  the  AOI  be? 

What  kind  of  blending  is  required  between  the  AOI 
and  the  IFOV?  How  large  should  the  IFOV  be?  How 
should  the  IFOV  be  blended  to  the  background?  How 
accurately  should  head  attitude  be  measured?  How 
accurately  should  eye  attitude  be  measured?  How 
quickly  should  the  display  stabilize  following  a 
head  or  eye  movement?  The  answers  to  these 
questions  were,  generally  not  available  in  the 
literature.  Information  regarding  perception 
thresholds  could  be  found  but  did  not  give  accept- 
ability  thresholds.  Accordingly,  several  experi¬ 
ments  were  devised  to,  at  least,  give  some  guidance 
in  designing  the  visual  simulation  system  hardware. 

Area  of  Interest 

Experiments  were  performed  to  get  an  idea  of  how 
large  tee  Area  of  Interest  (AOI)  had  to  be  in 
order  to  be  subjectively  acceptable  as  a  function 
of  the  delay  between  an  eye  movement  and  the  move¬ 
ment  of  the  AOI.  Note  that  the  movement  of  the  AOI 
consisted  of  a  movement  of  the  borders  of  the 
high  resolution  inset  with  no  change  in  the  appar¬ 
ent  location  of  image  features.  The  experimental 
apparatus  consisted  of  annular  projection  lens,  a 
variable  resolution  mask,  a  servo  system  to  rotate 
the  mask,  a  variable  delay  system,  and  an  eye 
tracker.  The  apparatus,  test  procedure,  and 
results  are  described  in  Reference  2,  and  pictured 
in  Figure  3.  In  carrying  out  these  experiments  it 
was  quickly  determined  that  hard  edges  (abrupt 
resolution  changes)  between  the  AOI  and  IFOV  were 
very  objectionable  and  distracting  to  most 
observers.  Consequently,  the  masks  were  fabricated 
to  cause  a  gradual  transition  of  resolution  rather 
than  an  abrupt  change.  Since  the  transition  region 
would  require  both  levels  of  resolution  the  size  of 
the  AOI  must  include  the  transition  region.  The 
results  of  the  eye  tracked  experiments  indicate 
that  an  AOI  width  of  25°  within  which  is  a  5°  wide 


Instantaneous  field  of  view  requirements  were 
determined  by  using  the  apparatus  pictured  in 
Figure  3  in  a  different  configuration.  The  eye 
attitude  sensor  was  removed  and  a  horizontal  head 
angle  sensor  substituted.  The  variable  resolution 
masks  were  replaced  with  servo  controlled  masks 
which  were  capable  of  providing  a  constant  resolu¬ 
tion  over  a  limited  field  angle.  Subjective 
evaluations  indicated  that  an  instantaneous  field 
of  view  of  130°  with  a  delay  of  80  milliseconds 
would  be  noticeable  but  not  objectionable. 
Experiments  using  a  head  slaved  instantaneous 
field  of  view  performed  on  ASPT  (3)  indicated  that 
a  field  width  of  90°  was  adequate  for  certain 
tasks.  Since  the  optical  design  of  the  display 
was  not  greatly  influenced  by  the  difference 
between  90°  and  130°,  it  was  decided  to  go  with 
the  wider  field  to  provide  peripheral  cues  for 
those  tasks  for  which  a  90°  field  might  not 
suffice.  The  blending  of  the  IFOV  to  the  back¬ 
ground  was  not  found  to  be  a  significant  problem. 
Hard  edges  of  the  IFOV  at  130°  were  just  noticeable 
anti  not  objectionable.  This  indicates  that  a 
smaller  field  with  seme  blending  may  suffice  but 
this  has  not  been  experimentally  verified. 


Image  Stability 


The  most  critical  performance  requirement  of  a 
helmet  mounted  display  is  to  provide  imagery 
which  is  acceptably  stable  against  head  movements. 
Experiments  were  performed  utilizing  a  head 
attitude  sensing  system  manufactured  by  Polhemus 
Model  SHMS  IIIA  to  provide  head  pointing  informa¬ 
tion  to  the  Visual  Technology  Research  Simulator 
(VTRS)  Computer  Image  Generator  (CIG) (4)  which 
then  provided  a  single,  monochrome  video  signal 
to  a  helmet  mounted  miniature  projection  CRT, 
manufactured  by  Systems  Research  Laboratories. 

The  projected  CRT  raster  was  reflected  from  a  1 
meter  radius  spherical  screen  coated  with  Scotch- 
lite  #7615  high  gain  screen  material  manufactured 
by  3M.  The  following  problems  were  noted: 
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Image  Lag 

The  thruout  delay  caused  by  the  head  attitude 
sensor  (16ms)  combined  with  the  CIG  computational 
thruput  delay  (50  milliseconds)  produced  a  lag  In 
proper  Image  positioning  perceived  as  an  angular 
Image  displacement  equal  to  the  angular  difference 
between  current  head  angle  and  the  head  angle  used 
to  compute  the  current  scene.  This  was  considered 
to  be  highly  unacceptable  and  led  to  a  display 
design  which  Incorporated  a  feature  to  compensate 
for  Image  lag  In  pitch  and  yaw.  Head  roll  rates 
were  found  to  be  sufficiently  slow  to  allow  an 
acceptable  lag  without  compensation  for  thrupur 
delay. 

Image  Jitter 

Although  the  specified  anglular  accuracy  (less 
than  1°)  of  the  SHMS  IIIA  was  adequate,  the 
precision  of  the  digital  signal  was  found  to 
produce  an  Image  jitter  of  approximately  0.1°. 

This  value  of  jitter  would  probably  be  acceptable 
for  a  wide  field  display  whose  resolution  Is  poor¬ 
er  than  0.1°,  but  since  the  resolution  goal  of  the 
display  system  Is  about  four  times  better  than 
0.1°  a  head  attitude  sensor  having  a  precision 
of  0.025°  would  be  required  If  system  resolution 
is  to  be  maintained. 

Image  Luminance  and  Contrast 

Luminance  values  usually  specified  for  outslde- 
the-cockplt  daylight  visual  simulation  displays 
are  typically  In  the  range  of  one  to  ten  foot- 
lamberts.  Accordingly,  a  display  brightness  of 
ten  foot-1  amberts  was  chosen  as  a  goal.  The 
helmet  mounted  laser  projector  configuration 
combined  with  a  retroreflectlve  screen  could 
provide  this  brightness.  The  retroflectlve  screen 
also  minimizes  cross  reflectance  problems  allow¬ 
ing  a  designed  contrast  ratio  of  thirty  to  one. 
However,  a  more  critical  problem  Is  the  contrast 
between  the  displayed  Image  as  seen  on  the  screen 
and  the  apparent  brightness  of  ghost  Imagery 
reflected  from  Inside  the  cockpit  surfaces.  This 
affect  was  noticeable  In  that  the  observer  had 
the  feeling  that  he  was  wearing  a  miner's  lamp  on 
his  helmet.  This  problem  was  noted  and  attempt 
has  been  made  to  resolve  It  by  designing  the 
screen  and  cockpit  surfaces  such  that  the  maximum 
luminance  of  ghost  images  would  be  less  than  the 
minimum  luminance  (dark  level)  of  the  displayed 
imagery  on  the  screen. 

Shadows 

Although  the  helmet  mounted  projector  was 
designed  to  cause  minimum  shadow  effects  the 
separation  of  the  projector  from  the  observer’s 
eyes  will  produce  residual  shadows  on  the  screen 
which  are  in  the  observer's  field.  Although  the 
magnitude  of  this  effect  was  computed.  Its  accept¬ 
ability  had  to  be  evaluated.  Accordingly  a  dis¬ 
play  configuration  was  assembled  which  produced 
the  same  type  of  shadows  as  would  be  apparent  In 
the  helmet  mounted  laser  projector  configuration. 
The  subjective  evaluation  using  this  apparatus 
Indicated  that  a  static  (head  not  moving)  situa¬ 
tion  the  shadows  of  struts  were  acceptable,  but 
that  head  motion  caused  objectionable  shadow. 

Thus,  It  was  decided  to  utilize  a  cockpit  config¬ 
uration  with  no  struts  within  the  available  field 
of  view.  Shadows  caused  by  the  cockpit  structure 


Itself  generally  lie  below  the  observer's  line  of 
sight  and  are  not  visible. 

Resolution 

The  resolution  capability  of  the  eye  peaks  at 
approximately  one  arc  minute  per  optical  line 
pair  for  foveated  high  contrast  targets  displayed 
at  a  luminance  of  10  foot- 1 amberts.  This  corres¬ 
ponds  to  an  acuity  of  2.0  or  capability  to  read 
the  20/10  line  on  a  Snellen  Eye  Chart.  However, 
the  specified  resolution  of  a  display  for  visual 
simulation  seldom  requires  this  demanding  perform¬ 
ance.  Typical  specifications  usually  correspond 
to  an  acuity  of  0.2  or  less.  An  acuity  of  0.2 
corresponds  to  a  limiting  resolution  of  10  arc 
minutes/optical  line  pair.  The  resolution  goal 
for  the  helmet  mounted  laser  projector  system 
was  not  determined  by  eye  capabilities  but  by  a 
computation  of  the  expected  resolution  obtainable 
with  a  nominal  1000  line/frame  raster  filling  the 
A0I.  Since  the  required  size  of  the  A0I  was 
roughly  25°  the  resolution  capability  Is  approx¬ 
imately  1.6  arc  minutes/TV  line  which  corresponds 
to  3.3  arc  minutes/TV  line  pair  or  five  minutes 
per  optical  pair.  This  resolution  Is  twice  as 
good  as  the  resolution  being  specified  In  some 
visual  simulation  systems  today.  To  fill  an 
available  field  of  view  240°H  x  180°V  with  a  non¬ 
head/eye  coupled  display  having  equivalent  resolu¬ 
tion  would  require  a  nominal  1000  line  raster  dis¬ 
play  for  each  25°  x  25°  segment  or  40  channels  of 
display/image  generator. 

COMPUTER  IMAGE  GENERATOR 

Since  the  primary  objective  of  the  helmet 
mounted  projector  project  was  to  demonstrate 
feasibility  of  the  concept,  the  effort  devoted  to 
the  Image  generator  was  limited  to  a  study  perform¬ 
ed  by  General  Electric  (5)  to  Investigate  mod¬ 
ifications  to  the  existing  VTRS  CIG  which  would  be 
required  to  demonstrate  and  evaluate  the  concept. 
The  results  of  this  study  Indicated  that  the 
following  modifications  would  be  required. 

Channel  Specific  Level  of  Detail 

Although  the  current  VTRS  CIG  has  the  capabil¬ 
ity  to  portray  a  given  feature  at  different 
levels  of  detail,  the  system  does  not  have  the 
capability  to  provide  different  levels  of  detail 
In  the  two  display  channels.  This  capability  Is 
essential  to  the  AOI-IFOV  concept  If  an  Increase 
In  apparent  Image  detail  Is  to  be  demonstrated, 
and  Its  effect  evaluated. 

Channel  Specific  Distortion  Correction 

The  existing  VTRS  CIG  has  the  capability  to 
provide  distortion  correction  whose  parameters 
can  be  varied  In  real  time  through  a  segmentation 
and  remapping  process  (6).  Modifications  would 
be  required  to  expand  this  capability  from  single 
channel  to  both  channels. 

Inset  Blending 

The  current  VTRS  CIG  does  not  have  the  capabil¬ 
ity  to  provide  an  Inset  AOI  which  smoothly 
transitions  to  the  IFOV.  A  scheme  for  accomplish¬ 
ing  a  blended  Inset  capability  Is  currently  being 
developed  for  AFHRL  at  Williams  AFB  for  evaluation 
of  a  dual  projector  concept  (7).  Such  a  scheme 
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would  be  required  for  the  helmet  mounted  laser 
projector  to  avoid  unacceptable  transition  between 
regions. 

Systems  Performance 

The  existing  VTRS  CIG  with  appropriate  modifica¬ 
tion  and  Interfaces  to  a  head  attitude  sensor  and 
an  eye  attitude  sensor  would  have  the  capability 
of  providing  a  displayed  scene  content  of  1,000 
potentially  visible  edges  In  the  1F0V  and  1,000 
edges  In  the  AOI.  The  apparent  edge  density  of 
the  entire  available  field  should  be  equivalent  to 
the  edge  density  observed  In  the  AOI  which  Is 
equivalent  to  a  perceived  total  of  40,000  edges 
although  the  verification  of  this  assumption  has 
not  been  accomplished, 

HEAD  ATTITUDE  SENSOR 

The  function  of  the  head  attitude  sensing 
system  Is  to  provide  a  head  pointing  direction  In 
pitch,  roll  and  yaw.  The  attitude  Information 
should  be  as  current  as  possible  and  precise  to 
0.025°  or  better.  A  head  attitude  sensing  system 
which  meets  the  precision  requirement  and  has  a 
thruput  delay  of  10  milliseconds  has  been  develop¬ 
ed  for  the  Aerospace  Medical  Research  Laboratory 
at  Wrlght-Patterson  AFB  by  Polhemus.  The  Polhemus 
system  employs  a  magnetic  field  radiator  mounted 
on  the  cockpit  structure  and  a  magnetic  sensor 
mounted  on  the  helmet.  The  principle  of  operation 
Is  discussed  In  Reference  8. 

Since  the  Polhemus  system  Is  the  most  likely 
candidate  for  Implementation  In  the  helmet  mounted 
laser  projector,  a  feasibility  experiment  utiliz¬ 
ing  a  single  channel,  monochrome  helmet  mounted 
laser  projector  together  with  the  VTRS  CIG  and  the 
retrof lectlve  screen  was  assembled  and  evaluated. 

The  feasibility  model  differed  from  the  final 
design  In  several  respects.  Only  one  frame  scan 
galvanometer  was  mounted  on  the  helmet  as  opposed 
to  three  galvanometers  in  the  helmet  mounted 
laser  projector  design.  An  off-the-shelf  fiber 
optics  array  was  utilized  to  relay  a  line  scan  to 
the  helmet  rather  than  a  custom  made  fiber  optics 
ribbon.  A  narrow  field  (40°)  off-the-shelf  projec¬ 
tion  lens  was  used  rather  than  the  140°  lens  call¬ 
ed  for  In  the  design.  The  results  of  this  experi¬ 
ment  indicated  that  the  galvanometer  caused  no 
noticeable  noise  In  the  head  attitude  sensor  as 
long  as  no  metallic  structure  got  between  the 
radiator  and  the  sensor  and  measurement  samples 
were  synchronized  to  occur  during  the  relatively 
quiescent  time  of  the  frame  scanner  (not  during 
flyback).  Although  the  magnetic  sensor  approach 
appears  to  be  viable,  alternative  head  attitude 
sensing  systems  were  also  considered.  The  best 
alternative  approach  studied  would  utilize  three 
automatic  polarlmeters  capable  of  slewing  at  head 
angular  rates  mounted  behind  small  holes  In  the 
screen  structure  and  polarized  suitably  coded, 
light  sources  on  the  helmet.  Automatic  polarl¬ 
meters  are  available  off-the-shelf  with  precision 
to  0.001°.  Unfortunately  slew  rates  are  on  the 
order  of  l°/second  rather  than  the  100°/second 
required  for  head  motions.  The  concept  of  utiliz¬ 
ing  automatic  polarlmetry  will  be  pursued  If 
required. 


EYE  ATTITUDE  SENSOR 

Many  techniques  for  monitoring  eye  movements 
have  been  developed  (9).  Unfortunately  no 
technique  Incorporates  all  of  the  desired  features 
of  an  eye  tracker  for  the  helmet  mounted  laser 
projector.  Electroculagraphy  (EOG)  has  the  desired 
measurement  range  (to  the  limit  of  eyeball  rotation) 
and  causes  no  obstruction  of  the  field  of  view. 

But  EOG  Is  noisy  and  highly  sensitive  to  electrode 
contact,  facial  muscle  activity,  and  light  adapta¬ 
tion  level.  Remote  oculometers  are  limited  In 
measurement  range  to  approximately  +30°,  have 
relatively  slow  response  (due  to  frame  rate  of 
sensor),  and  require  the  observer  to  keep  his  head 
pointed  toward  the  oculometer.  Helmet  mounted 
oculometer  configurations  are  possible  but  the 
advantage  of  free  head  movement  Is  offset  by  the 
requirement  for  a  beamsplitter  and  supporting 
structure  within  the  observer's  field  of  view.  A 
limbus  tracking  system  Is  restricted  to  a  measure¬ 
ment  range  of  +20°  and  Is  also  obtrusive  Into  the 
observer's  fleTd  of  view.  However  the  limbus 
tracker  has  relatively  fast  response  and  Is 
relatively  Inexpensive.  A  limbus  tracker  was 
utilized  In  the  eye  tracker  AOI  experiments 
described  above. 

Relative  Head-Helmet  Motion 

The  question  of  relative  movement  between  the 
observer's  head  and  his  helmet  Is  not  critical  to 
the  stability  of  the  display  since  the  helmet  Is 
tracked  and  the  projector  Is  mounted  on  the  helmet. 
However,  any  eye  tracking  system  which  measures 
eye  attitude  In  relation  to  a  monitoring  device 
fixed  on  the  helmet  will  be  affected  by  this 
relative  movement.  An  experiment  was  designed  and 
carried  out  to  measure  this  relative  motion.  The 
apparatus  consisted  of  a  custom  molded  bite  fixture 
and  a  Navy  aviator's  helmet  Model  APH-6.  A  rigid 
conducting  bar  extended  from  the  bite  fixture  to 
the  brow  area  on  the  subject's  head.  The  bar  was 
centered  In  an  adjustable  gap  between  two  contact 
points  and  wired  such  that  contact  between  the 
bar  and  either  one  of  the  sides  of  the  gap  would 
cause  a  battery  powered  lamp  to  light.  The  results 
of  this  experiment  Indicated  that  head  rotations 
In  yaw  at  rates  less  than  60°/second  caused 
relative  movements  of  less  than  0.010  Inches. 

Higher  head  rates  or  head  roll  caused  relative 
movements  of  less  than  0.025  Inches.  These  values 
can  be  real  ted  to  eye  movement  accuracies.  The 
movement  of  the  limbus  of  the  eye  Is  approximately 
0.010  Inches  per  degree  of  eye  rotation.  Corneal 
relfex  motion  is  approximately  0.003  Inches  per 
degree  of  eye  rotation.  Although  no  attempt  was 
made  to  custom  fit  the  helmet  or  otherwise  stabil¬ 
ize  It  beyond  the  normal  chin  strap  the  relative 
motion  was  within  an  acceptable  range  for  a  helmet 
mounted  limbus  tracker  but  not  acceptable  for  a 
helmet  mounted  corneal  reflex  tracker.  However, 
oculometers  have  been  developed  which  utilize  the 
pupil  location  as  a  reference  (10).  Such  systems 
are  limited  to  frame  rate  response  since  the  whole 
Image  of  the  eye  must  be  processed  to  determine 
the  location  of  the  corneal  reflex  as  well  as  the 
eye  pupil. 

Eye  Position  Prediction 

Rapid  eye  movements  (called  saccades)  have  a 
characteristic  motion  which  allows  prediction  of 
the  endpoint  when  the  movement  Is  only  halfway 
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completed  (11).  Although  such  prediction  (in  order 
to  get  a  head  start  on  generating  the  Imagery  tor 
the  new  AOI)  has  not  been  determined  to  be  required 
It  certainly  would  be  desirable  If  It  could  be 
efficiently  Implemented.  Figure  4.  shows  three 
plots.  The  top  plot  shows  the  output  of  an  analog 
eye  tracker,  such  as  a  limbus  tracker.  The  first 
part  of  the  curve  contains  a  20°  saccade  having 
a  duration  of  60  milliseconds.  The  last  part  of 
the  curve  shows  the  effect  of  a  blink.  The  middle 
curve  shows  the  velocity  as  a  function  of  time 
obtained  by  differentiating  the  angle  curve.  Note 
that  the  saccade  shows  a  peak  velocity  halfway 
through  the  saccade.  The  velocity  profile  for  the 
blink  Is  also  depicted  .  The  lower  curve  represents 
the  output  of  a  predictor  device  developed  under  a 
contract  with  Carnegle-Mellon  University.  The 
predictor  Is  capable  of  predicting  the  final  eye 
position  by  measuring  the  time  at  which  velocity 
peaks  and  then  doubling  the  angle.  The  predictor 
can  also  discriminate  against  eye  blinks  by 
utilizing  an  algorithm  which  contains  velocity 
thresholds  and  eye  movement  monitor  characteristics . 
The  net  time  savings  In  this  example  Is  30  milli¬ 
seconds.  Longer  saccades  would  result  In  greater 
time  savings.  Indications  are  that  prediction 
accuracies  of  2°  are  obtainable  for  saccades  of 
20°. 

DISPLAY  SYSTEM 

A  description  of  the  design  and  operation  of  the 
display  system  has  been  presented  previously  (1) 
and  will  not  be  repeated  here.  What  was  not 
discussed  In  the  previous  paper  were  some  of  the 
design  Issues  and  tradeoff  analyses  which  led  to 
the  system  design.  A  large  part  of  this  effort 
was  performed  by  Dan  Lobb  under  a  contract  with 
the  University  of  Central  Florida. 

Line  Image  Generator 

The  functions  of  the  line  image  generator  are 
to  provide  sufficient  three  color  laser  light, 
to  provide  two  -  three  color  modulated  beams,  to 
provide  scanning  for  both  beams.  The  issues  were: 
What  laser  or  laser  mix  would  be  optimum  in 
terms  of  available  colors,  power,  and  reliability? 
What  type  of  optics  would  be  most  desirable  for 
the  color  separation  and  recombination?  What 
type  of  modulators  sh'jld  be  used?  What  type  of 
scanners  should  be  used?  Figure  5  shows  a 
schematic  diagram  of  the  line  image  generator. 

The  answers  to  these  questions  were  primarily 
based  on  our  laboratory's  experience  with  laser 
display  systems. 

Lasers 

Our  experience  with  a  multi-laser  display 
system  and  the  problems  associated  with  reliability 
and  maintainability  led  to  a  requirement  to  use 
as  few  lasers  as  possible.  A  colorimetric  analysis 
indicated  that  a  single  10  watt  Argon  Ion  Laser 
would  provide  sufficient  luminance  In  blue  and 
green  plus  enough  excess  light  to  pump  a  red  dye 
laser.  Based  on  desired  display  luminance  and 
computed  losses  between  the  laser  and  the  screen, 
the  analysis  concluded  that  the  latest  light 
required  is  approximately  1,000  lumens  in  wave¬ 
lengths  actually  used  after  any  necessary  loss 
from  some  wavelengths  to  achieve  a  good  white. 

The  composition  of  the  laser  white  is:  Red  primary 
(from  the  Rhodamine  6-G  dye  cell)  having  a  wave¬ 


length  of  610  nanometers  and  power  of  1,300  mill- 
watts  ;  a  green  primary  of  514.5  nanometers  and 
power  of  1,500  milliwatts  (about  half  of  the  green 
line  power  directly  from  the  10  watt  Argon  Laser); 
and  a  blue  primary  having  a  dominant  wavelength 
of  470  nanometers  and  power  of  1,400  milliwatts 
(composed  of  the  short  wavelength  outputs  of  the 
Argon  Laser  from  454  nanometers  to  476  nanometers). 
The  remaining  Argon  power  Is  utilized  to  pump  the 
dye  coll.  Thus  the  problems  associated  with 
multiple  lasers  can  be  avoided  . 

Color  Splitting 

There  are  two  practical  options  for  separating 
the  Argon  Laser  output  into  the  various  colors 
required:  Dispersive  prisms  and  dichroics.  The 
problems  of  specifying  and  manufacturing  dichroics 
to  separate  wavelengths  as  close  as  the  488  nano¬ 
meter  Argon  Laser  line  (used  to  pump  the  dye)  from 
the  476  nanometer  Argon  line  (which  provides  a 
large  fraction  of  the  blue  primary).  On  the  other 
hand,  dichroic  splitting  Is  simple  and  straight¬ 
forward.  After  a  careful  weighing  of  advantages 
and  disadvantages  the  prism  dispersion  method  was 
chosen  as  the  preferred  technique  . 

Modulation 

At  the  video  bandwidths  of  Interest, acousto 
optic  modulators  offer  the  most  efficient,  cost 
effective  method  for  intensity  modulating  the 
six  beams  of  laser  light  resulting  from  the  color 
splitting  components  (2  channels  of  3  primaries 
each) . 

Color  Combination 

Since  the  six  modulated  beams  must  be  recombined 
prior  to  l<ne  scanning  as  two  beams  the  choice  of 
combining  technique  must  be  made.  In  this  case 
there  is  significant  separation  between  the 
primaries  (the  closest  being  the  514.5  green  and 
476  blue)  allowing  the  simplicity  of  dichroics  to 
be  preferred. 

Line  Scanner 

The  choice  of  a  line  scanning  system  was  almost 
forced.  Acousto-optic  techniques  would  have 
required  six  independent  line  scanning  channels 
with  obvious  problems  of  balancing  and  registra¬ 
tion.  On  the  other  hand  a  rotating  polygon  system 
could  scan  both  three  color  beams  simultaneously. 

Fiber  Optics  Relay 

The  function  of  the  fiber  optics  relay  is  to 
transmit  the  two  three  color  laser  scan  lines  to 
the  helmet.  The  basic  problem  associated  with 
this  arrangement  is  avoiding  image  artifacts 
caused  by  broken  fibers  or  different  transmission 
through  different  fibers.  Several  experiments 
were  performed  to  evaluate  the  effect  of  broken 
fibers  and  to  minimize  the  effect  of  different 
transmissions.  The  conclusion  was  that  even  a 
single  broken  fiber  was  immediately  obvious  in 
the  display  but  its  effect  on  training  performance 
could  not  be  predicted.  A  specification  for  a 
fiber  bundle  containing  no  broken  fibers  was 
prepared.  As  of  this  writing  two  manufacturers 
are  under  contract  to  provide  such  bundles  for 
test  and  evaluation  with  delivery  expected  in 
August  1981.  The  apparent  transmission  of 
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different  fibers  was  found  to  be  strongly  Influ¬ 
enced  by  the  collecting  aperture  used  to  gather 
light  at  the  output  of  the  bundle.  For  the 
specific  fiber  array  tested,  It  appeared  that  a 
collecting  aperture  of  f/5  would  suffice. 

Helmet  Mounted  Projector 

The  functions  of  the  helmet  mounted  projector 
are:  To  offset  the  line  scans  In  the  line 
direction  to  follow  eye  movements  and  compensate 
for  rapid  head  yaw  motion;  provide  frame  scanning 
for  both  rasters;  provide  offset  capability  in 
the  cross  line  direction;  provide  composite 
frame  from  two  Independent  Images;  and  project 
the  composite  frame  onto  the  screen.  A  schematic 
diagram  of  the  helmet  mounted  optics  Is  pictured 
In  Figure  6.  The  design  was  developed  under  the 
rather  severe  constraint  of  having  to  be  head 
supported  and,  at  the  same  time,  composed  of 
components  which  were  either  off-the-shelf  or 
represented  low  risk  development.  All  of  the 
above  requirements  were  met  by  this  design. 

Screen 


The  requirements  for  the  display  screen  para¬ 
meters  are  driven  by  two  constraints;  the  con¬ 
trast  between  Images  observed  on  the  screen 
surface  and  reflected  off  Inside-the-cockplt 
surfaces  should  be  high,  a, id  the  screen  structure 
should  be  an  existing  10  foot  radius  dome.  By 
painting  all  inside  -  the  -  cockpit  surfaces 
flat  black  and  tilting  all  specular  surfaces  such 
that  no  specular  reflections  can  be  directed 
toward  the  observer's  head  the  interior  of  the 
cockpit  can  be  assumed  to  be  a  screen  having  a 
gain  of  0.1  or  less  located  approximately  two 
feet  from  the  observer  .  This  implies  that  the 
screen  gain  required  to  keep  inside  the  cockpit 
Imagery  luminance  below  the  dark  level  of  the 
display  (nominally  3%  of  peak  brightness)  must 
be  greater  than  75  .  Off-the-shelf  retroreflective 
screen  materials  were  experimentally  evaluated 
with  the  results  indicated  In  Figure  7  .  The 
three  sets  of  data  represent  gai  n  measurements 
of  Avery  International  Retroref lector  (embossed 
corner  cubes);  3M  Scotchlite  Type  7615  and  3M 
Scotchlite  Type  8910.  The  results  of  the  evalua¬ 
tion  of  the  embossed  corner  cube  material  are 
somewhat  misleading  since  this  material  was  not 
uniform  and  since  it  displayed  a  six-lobed 
retroreflective  return  pattern  when  Illuminated 
with  laser  light.  The  conclusion  was  that 
Scotchlite  coating  8910  performed  adequately 
well  over  the  range  of  angles  required  by  the 
helmet  mounted  laser  projector  (approximately 
0.5°  to  1.5°  projection  point  eye  point  separa¬ 
tion)  .  However,  a  more  uniform  gain  character¬ 
istic  could  be  obtained  by  modifying  the  index 
of  refraction  of  the  glass  beads  utilized  to 
manufacture  the  screen.  A  contract  study  with 
the  Optical  Sciences  Center  at  the  University  of 
Arizona  resulted  In  the  conclusion  that  an  Index 
of  refraction  of  approximately  1.87  would  result 
in  a  more  uniform  distribution  over  the  desired 
range.  Preliminary  discussions  were  held  with  3M 
which  Indicated  that  such  a  specification  was 
feasible  within  the  constraints  of  the  manufactur¬ 
ing  processes  utilized  for  their  standard  products. 


described  In  thts  paper  represent*  en  overview  of 
an  exploratory  development  effort  which  has  culmin¬ 
ated  In  a  specification  for  a  visual  simulation 
system  which  offers  great  potential  for  Improved 
performance  at  low  cost  when  compared  to  conven¬ 
tional  mosaic  approaches  to  the  wide  field  high 
resolution  display  problem.  Based  on  this  effort 
an  advanced  development  program  has  bee  n  Initiated 
which  will  result  In  the  fabrication  of  a  research 
tool  Incorporating  the  design  concepts  outlined 
In  this  paper.  The  research  tool  will  be  Integrat¬ 
ed  Into  the  Visual  Technology  Research  Simulator 
Facility  at  NAVTRAEQUIPCEN  for  evaluation  of 
technical  performance  and  training  effectiveness. 
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Figure  1.  Display  Optical  System 
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Figure  5.  Line  image  Generator 
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COMPUTER  PROGRAM  FOR  DISTORTION  ANALYSIS 
IN  SPHERICAL  SCREEN  DISPLAYS 

Richard  C.  Hebb 


Simulation  Technology  Branch 
Naval  Training  Equipment  Center 
Orlando,  Florida 

ABSTRACT 

In  visual  simulation,  the  distortion  of  imagery  in  wide-angle  display  systems  is  a  major  concern.  Effective  flight  training  re¬ 
quires  that  imagery  presented  to  a  trainee  provide  a  proper  perspective  view  of  his  simulated  environment  without  distortion. 
Use  of  spherical  screens  (domes)  Introduces  both  perspective  and  geometrical  distortion  into  the  wide-angle  displays.  Use  of 
video  projection  systems  with  Computer  Image  Generation  (CGI)  offers  the  options  of  raster  shaping  or  computer  re-mapping 
of  raster  pixels  for  distortion  correction.  The  goal  in  distortion  correction  is  to  provide  proper  perspective  of  imagery  to  a 
trainee.  The  basic  causes  for  distortion  and  a  computer  program  for  analysis  of  spherical  screen  distortion  will  be  discussed. 


INTRODUCTION 

Visual  Flight  Simulation 

Visual  flight  simulators  are  being  developed  into  an  important  part 
of  the  training  that  pilots,  both  commercial  and  military,  are  receiving 
for  development  and  upkeep  of  their  flight  skills.  The  ultimate  goal  in 
visual  simulation  is  to  provide  a  realistic  view  of  the  environment  about 
a  simulated  aircraft  to  increase  the  effectiveness  of  training  exer¬ 
cises.  (1)  A  trainee’s  view  of  this  environment,  in  conjunction  with 
mechanical  simulation  of  the  aircraft  dynamics  and  structure,  can  in¬ 
duce  many  physical/psychological  effects  of  actual  flight.  (2)  Imagery 
is  often  provided  via  computer  image  generation  (CIG)  and  displayed 
by  video  projection  systems.  Generally,  the  CIG  system  takes  account 
of  viewpoint  and  heading  direction  within  a  mathematically  modeled 
landscape  (database)  to  generate  a  view  of  this  database  during  a 
simulated  flight.  (3)  The  effectiveness  of  the  visual  simulation  depends 
on  many  factors,  including  the  Field  Of  View  (FOV),  detail  in  the 
database,  resolution  capability,  display  brightness  and  contrast,  and 
relative  distortion  of  the  imagery. 

Wide-Angle  Visucl  Displays 

The  goal  of  realism  in  visual  simulation  has  led  to  the  use  of  very 
wide-angle  displays  filling  a  horizontal  FOV  of  90  °  or  more  at  the  pilots 
viewpoint.  Increasing  the  FOV  to  greater  than  180°  has  led  to  the  use  of 
spherical  screens  (domes)  with  a  number  of  projectors  filling  different 
parts  of  the  pilot's  available  FOV  to  form  a  wide-angle  scene.  (4)  Ideal¬ 
ly,  the  projectors  and  viewpoint  should  be  located  at  the  center  of  the 
dome,  or  at  least  at  the  same  point,  to  reduce  distortions  for  the  viewer. 
Unfortunately,  physical  restrictions  do  not  allow  a  number  of  projec¬ 
tors  and  the  viewer  to  occupy  the  same  position  in  space,  thus  forcing 
the  oblique  projection  of  imagery  onto  the  dome.  This  fact  results  in 
distortion  of  the  imagery. 
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Flgura  1 .  Mapping  via  TangantS  Lana,  Non-Dtatortad 


Distortion 

Distortion  refers  to  the  geometry  of  an  image  as  compared  to  the  ac¬ 
tual  geometry  of  the  objects  involved.  Perhaps  the  best  way  to  describe 
the  concept  of  distortion  as  related  to  optical  systems  is  to  first  consider 
the  concept  of  a  distortionless  lens  mapping.  Figure  1  shows  the  map¬ 
ping  process  of  a  well  corrected  F-Tan  9  lens  as  a  rectangular  object  is 
mapped  through  the  lens.  The  resulting  image  formed  is  again  a  rec¬ 
tangular  figure.  The  radial  distance,  R,  to  an  image  point  is  determined 
by  the  tangent  of  the  angle,  9  ,  that  an  object  point  subtends  from  the 
optical  axis  of  the  lens  system.  Hence  the  lens  mapping  equation, 
R  =  F-Tan  6  ,  where  F  is  the  focal  length  of  the  lens.  If  the  direction  of 
the  mapping  process  is  reversed,  we  then  have  the  case  of  the  lens  being 
used  as  a  projection  lens  versus  use  as  a  taking  lens.  For  this  case,  any 
imagery  placed  on  the  plane  denoted  as  the  image  plane  will  be  transfer¬ 
red  to  the  object  plane  without  distortion.  This  is  the  concept  of  a 
distortionless  lens  mapping. 

Next,  we  should  consider  the  concept  of  distortionless  viewing  of  a 
projected  image.  If  it  were  possible  to  plac.  the  eye  of  a  viewer  at  the 
exit  pupil  of  a  projection  lens  which  has  an  F-Tan  9  mapping  function, 
as  in  figure  2,  then  the  viewer  would  perceive  no  distortion  of  the 
imagery  projected  from  the  image  plane.  However,  if  the  viewer  is 
removed  from  the  exit  pupil  location,  then  the  shape  of  a  rectangular 
object  would  no  longer  appear  rectangular.  This  removal  of  the  view¬ 
point  from  the  exit  pupil  results  in  a  form  of  distortion  known  as 
Perspective  Distortion,  which  usually  results  in  rectangular  objects  ex¬ 
hibiting  a  keystone  shape.  The  key  to  reducing  perspective  distortion  is 
to  place  the  exit  pupil  of  the  projector  as  close  as  possible  to  the 
viewer’s  eyepoint. 
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Figure  2.  Dtstorttoniesa  Viewing 


In  a  Jome  display  system,  their  exists  another  form  of  distortion. 
This  form  is  referred  to  as  Geometric  Distortion,  which  involves  the 
oblique  projection  of  imagery  onto  screen  surfaces  which  are  not  flat 
display  planes.  The  result  is  that  the  projection  of  straight  lines  onto  the 
screen  surface  are  viewed  as  curved  lines  by  the  observer.  It  should  be 
noted,  that  if  the  viewpoint  and  exit  pupil  of  a  F-Tan  9  lens  are  coinci¬ 
dent,  then  the  shape  of  the  screen  cannot  contribute  to  distortion.  (7) 


Therefore,  the  amount  of  distortion  depends  on  the  size  and  shape  of 
the  screen,  as  well  as  projector/viewpoint  positioning.  As  the  projector 
and  viewpoint  are  displaced  from  each  other  the  apparent  size  and 
shape  of  the  projected  imagery,  as  well  as  the  angular  subtense  and 
position,  will  vary  accordingly.  It  is  the  purpose  of  the  computer  pro¬ 
gram  to  be  described  to  consider  these  two  forms  of  distortion,  that  Is 
Perspective  and  Geometric  Distortion,  in  spherical  screen  display 
systems. 

Distortion  definitions  vary  according  to  the  type  of  projection  system 
involved.  The  method  of  calculating  distortion  in  this  paper  is  based  on 
the  Institute  of  Radio  Engineers  (IRE)  Standards  for  Television.  (J) 
This  method  defines  the  Geometric  Position  Error  (OPE)  for  any  point 
in  the  field  as  the  magnitude  of  the  distance  from  the  point  to  its  ideal 
location.  This  implies  a  radial  distance  measurement  from  the  ideal 
location  of  a  point  to  its  actual  position.  The  percentage  of  distortion  is 
then  found  by  dividing  the  GPE  by  the  full  field  height  of  the  image. 
Figure  3  shows  the  linearity  chart  given  in  the  IRE  standards  for 
distortion  measurements.  This  chart  is  placed  over  a  video  monitor 
which  has  an  alignment  pattern  generated  by  a  test  signal  generator  and 
qualitative  measurements  are  made  by  observing  the  position  of  the 
pattern  with  reference  to  calibrated  circles. 
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Figure  3.  WE  Linearity  Chart 


Distortion  Correction 

The  first  attempts  at  correcting  distortion  primarily  involved  the  use 
of  optical  elements  to  pre-distort  the  image  upon  projection,  resulting 
in  an  image  that  appeared  non-distorted  to  the  viewer.  This  method, 
although  successful,  required  the  correction  to  be  physically  set  at  the 
time  of  lens  design:  for  the  display  system.  The  capability  to  easily 
change  the  correction  factor  was  lost  along  with  the  ability  to  use  off- 
the-shelf  standard  lenses. 

With  the  use  of  video  projection  systems,  the  option  of  correcting 
distortion  by  altering  the  scanning  geometry  of  the  video  raster  (raster 
shaping)  became  available.  In  this  way  the  image  can  be  pre-distorted 
before  the  projection  lens  in  order  to  produce  a  non-distorted  view.  (6) 
The  advent  of  Computer  Image  Generators  brought  an  alternate 
method  for  corrections.  The  objects  to  be  projected  are  remapped  in 
the  CIO  computation  to  provide  the  required  object  pre-distortion 
before  being  placed  on  the  video  raster  for  projection.  (7)  Raster  shap¬ 
ing  and  CIG  remapping  may  be  combined  to  reduce  the  complexity  of 
the  individual  corrections.  Both  raster  shaping  and  CIG  remapping  will 
allow  changes  in  corrective  action  to  some  degree  and  may  also  allow 
dynamic  correction. 

Lens  Mapping 

In  the  computer  program  developed  for  analysis  of  distortion,  there 
are  four  types  of  lens  mappings  considered  for  the  projection  lens. 
These  four  lenses  allow  the  analysis  to  include  consideration  of  the 
effects  of  the  lens  mappings  on  the  Anal  net  distortion.  The  lens  must 
be  included  in  the  analysis  for  it  is  an  integral  part  of  the  system.  The 
four  lenses  are: 

1 .  F-Tan  9  (distortionless  lens) 

2.  F-Tan  9  with  primary  distortion 

3.  F -9  (9  in  radians) 

4.  F-Sin  9 


All  these  lens  mappings  imply  a  radially  symmetric  mapping  with  the 
center  of  the  mapping  plane  on  the  opticid  axis  of  the  system.  The 
F-Tan  9  lens  places  an  image  point  on  the  image  plane  according  to  the 
tangent  of  the  angle  ( 9 )  between  the  optical  axis  and  the  object  point. 
The  mapping  equation  is: 

Rt  -  F-Tan  (8 ), 

where  Rt  is  the  image  point  radial  distance  from  the  center  of  the 
image  plane,  F  is  the  lens  focal  length,  and  9  is  the  angle. 

The  F-Tan  9  with  primary  distortion  (F-Tan  9'  P)  lens  is  defined  as 
a  departure  from  the  F-Tan  $  mapping  due  U  an  approximation  of 
Tan-  9  by  only  two  terms  of  a  power  series.  The  resulting  equation  is: 

Rp  -  Rt  (1  +(DFACTOR  •  Rt*)), 

where  Rp  is  the  image  point  radial  distance  for  the  F-Tan  9  •  P  map¬ 
ping,  Rt  is  the  radial  distance  for  the  F-Tan  9  mapping,  and 
DFACTOR  is  the  primary  distortion  factor. 

An  F-  9  lens  maps  object  space  to  image  space  according  to  the  angle 
in  radians  to  the  object  point,  resulting  in  the  radial  position  of  the 
image  (Rg  )  being  defined  as: 

Rfl  -  F*. 

The  F-Sin  9  lens  implies  a  mapping  according  to  the  sine  of  the  op¬ 
tical  axis,  yielding  the  mapping  equation: 

Rs  =  F  Sin  ( 9 ). 

Figure  4  shows  the  relative  distortions  of  a  rectangular  object  by  the 
lenses  previously  mentioned 
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Figure  4.  Relative  Distortions  of  a  Rectangular  Ob)eot  by  Various  Lens 
Mappings 


THE  PROGRAM  -  MAPTAO 

Introduction 

MAPTAG  is  an  acronym  for  Mapping  Tables  and  Graphs,  a  pro¬ 
gram  written  in  FORTRAN  for  distortion  analysis.  The  facilities  for 
development  and  operation  of  the  program  are  part  of  the 
NAVTRAEQUIPCEN’s  Computer  Simulation  Laboratory.  The  com¬ 
puter  system  utilized  is  a  VAX-11/780  with  graphics  provided  via  a 
Tektronix  graphic  terminal  model  4014-11. 
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The  program  is  designed  to  find  the  required  distorted  raster  shape 
for  projection  onto  a  dome  from  a  particular  projection  point.  This 
projected  raster  is  to  be  viewed  at  a  viewpoint  as  a  non-distorted  raster, 
The  location  of  the  viewpoint  and  projector  can  be  located  inside  or 
outisde  the  dome  of  radius  R.  The  raster  is  placed  on  a  View  Window  of 
variable  height  and  width  in  degrees,  and  can  be  centered  at  any  loca¬ 
tion  on  the  dome. 

In  order  to  describe  the  projection/viewing  system,  the  location  of 
the  projector,  viewpoint,  and  Image  points  are  referenced  to  a  3-D 
coordinate  system  located  at  the  center  of  the  dome  (Figure  5).  In  this 
system,  the  Z-axis  is  positive  upwards,  the  X-axis  is  positive  forward, 
and  the  Y-axis  is  positive  to  the  left.  Additionally,  the  angles  for  projec¬ 
tion  and  viewing  are  spherical  angles  referenced  to  the  positive  X-axis. 
Vertical  angles  are  positive  above  the  dome  horizon  and  negative  below 
the  horizon  with  a  maximum  magnitude  of  90°.  Horizontal  angles  are 
positive  for  a  counter-clockwise  rotation  (positive  X-axis  into  the 
positive  Y-axis)  when  viewed  from  a  point  on  the  positive  Z-axis.  The 
hoiizontal  angles  have  a  maximum  magnitude  of  180°. 


Flour*  5.  3-D  Coordinate  System  For  Spherical  Screen 


There  are  three  other  coordinate  systems  involved  in  the  program. 
These  are  two  3-D  coordinate  systems,  located  at  the  viewpoint  and 
projection  point,  and  a  2-D  coordinate  system  used  to  define  the  Input 
Plane.Each  of  the  3-D  systems  are  parallel  to  the  other,  with  the  2-D 
system  transformed  to  the  sphere-centered  3-D  system  as  the  Input 
Plane  becomes  the  View  Window. 

View  rVindow 

The  View  Window  is  defined  as  that  part  of  the  available  FOV  that  is 
being  filled  by  one  projector.  Ideally,  the  viewer  will  see  objects  pro¬ 
jected  onto  this  window  as  non-distorted.  If  a  video  projection  system 
is  used,  then  the  viewer  will  want  to  see  a  raster  plane  that  has  pixels  at 
equal  increments  across  the  raster  lines  and  equally  spaced  raster  lines 
on  the  window.  This  is  the  Input  Plane  and  is  constructed  by  consider¬ 
ing  the  desired  angular  height  and  width  as  well  as  the  position  of  the 
center  of  the  view  window  (FCTR)  relative  to  the  viewpoint.  These 
values  are  used  to  find  the  height  and  width  of  the  raster  plane  in 
dimensional  units  that  are  fixed  with  respect  to  dimensional  units  used 
for  all  3-D  locations. 

The  Input  Plane  has  a  2-D  coordinate  system  with  its  origin  set  at  the 
center  of  the  plane  (Figure  6).  In  this  coordinate  system  the  Y-axis  is 
positive  upward  and  the  X-axis  is  positive  to  the  left.  Once  the  height 
and  width  of  the  Input  Plane  are  known,  then  the  first  point  on  the 
plane  is  found  by  dividing  the  height  and  width  in  half.  This  first  point 
is  defined  as  the  top  left  point  on  the  plane.  Subsequent  points  on  the 
plane  are  found  by  considering  the  number  of  points  across  the 
horizontal  raster  lines  (NHORIZ)  and  the  number  of  vertical  raster 
lines  (NVERT)  on  the  plane.  The  width  of  the  plane  is  divided  by 
(NHORIZ- 1)  to  find  the  linear  increments  along  the  raster 
(HINCRFMENT),  while  the  height  is  divided  by  (NVERT-1)  to  find 
the  linear  increments  between  raster  lines  (VINCREMENT).  To  find 
the  next  point  along  a  raster  line  on  the  plane,  the  HINCREMENT  is 
subtracted  from  the  previous  points’  X-coordinate  with  the 


Y-coordinate  remaining  the  same.  At  the  end  of  a  raster  line,  the 
VINCREMENT  is  subtracted  from  the  previous  points’  Y-coordinate, 
and  the  X-coordinate  is  reset  to  the  X-coordinate  of  the  first  point.  In 
this  way,  raster  lines  are  drawn  from  left  to  right  and  top  to  bottom  on 
the  Input  Plane. 
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Figure  0.  3x3  Input  Plane 
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After  a  point  on  the  Input  Plane  is  found,  the  Input  Plane  is  aligned 
to  be  normal  to  the  viewer's  Line  of  Sight  (LOS)  and  translated  to  the 
desired  View  Window  center  (Figure  7).  At  this  point,  the  View  Win¬ 
dow  is  projected  onto  the  surface  of  the  spherical  screen  but  still  ap¬ 
pears  to  be  a  flat  non-distorted  raster  from  the  viewpoint. 


Spherical  Screen  Projection 

To  find  the  intersection  of  a  ray  projected  from  the  viewpoint 
towards  the  spherical  screen,  the  subroutine  Sphere  Point  from  Angles 
(SPTFANG)  is  used.  This  subroutine  uses  spherical  angles  of  projec¬ 
tion  to  define  the  direction  of  individual  rays  at  the  viewpoint.  This 
distance  from  the  viewpoint  to  the  screen  is  calculated  and  used  to  find 
the  terminus  of  the  ray  in  3-D  coordinates. 

The  angles  to  points  on  the  View  Window  plane  from  the  viewpoint 
are  found  by  subroutine  Spherical  Angles  from  Points  (SANGFPT) 
and  then  are  input  to  SPTFANG.  The  View  Window  is  then  mapped 
onto  the  screen  surface  and  we  are  now  ready  to  find  a  perspective  view 
of  the  View  Window  from  the  projection  point. 

Perspective  View 

Another  subroutine,  PSPECTIVE,  is  used  to  find  the  perspective 
view  of  the  View  Window  from  the  projection  point.  Initially  the 
subroutine  translates  the  origin  of  the  3-D  coordinates  for  the  View 
Window  to  the  projection  point.  Then,  the  View  Window  is  rotated 
about  the  projector  to  align  the  center  of  the  View  Window  (PCTR) 
with  the  projector’s  X-axis.  (8)  Having  accomplished  these  operations, 
the  tangents  to  points  on  the  View  Window  can  be  found  relative  to 
PCTR. 
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Figure  9'  Protection/ Viewing  Diagram 


PSPECTIVE  allows  this  tangent  mapping  to  be  altered  by  a  remap¬ 
ping  through  one  of  three  other  lens  mappings  listed  previously.  Only 
the  tangent  mappings  will  give  a  correct  perspective  from  the  projection 
point,  with  the  three  additional  mapping  choices  distorting  the  view 
according  to  their  mapping  functions.  These  mapping  coordinates  are 
sent  to  plotting  programs  which  draw  the  View  Plane  on  a  graphic  ter¬ 
minal  with  hard-copy  available. 

Reference  circles,  indicating  total  FOV’s  of  90°  and  110°,  are 
included  on  the  output  mappings.  These  FOV’s  are  generated  by  subroutine 
FOV  and  are  dependent  on  the  type  of  lens  mapping  specified.  These 
reference  circles  can  be  used  to  determine  the  location  of  object  points 
relative  to  the  optical  axis  of  the  lens.  Points  that  lie  on  a  circle  are 
defined  as  being  located  at  the  half-angle  of  the  total  FOV.  For  exam¬ 
ple,  if  a  point  is  on  the  90°  FOV  circle,  then  the  angle  between  the  op¬ 
tical  axis  and  the  ray  to  that  point  is  45°.  Points  outside  the  FOV 
reference  circle  are  at  greater  angles,  while  points  inside  the  FOV 
reference  angles  are  at  an  angle  less  than  the  FOV  half-angle.  The  FOV 
reference  circles  give  an  excellent  way  to  approximate  the  total  FOV  re¬ 
quired  for  a  particular  projection  arrange. sent. 

Mapping  Output 

Figure  8  shows  the  basic  mapping  processes  for  MAPTAO  contain¬ 
ing  two  mappings.  The  first  maps  the  input  plane  on  to  the  surface  of 
the  dome  to  provide  a  non-distorted  raster  to  the  viewer.  This  raster 
pattern  on  the  dome  is  the  View  Window.  In  the  process  of  creating  the 
View  Window,  a  Comparison  Plane  is  also  formed  which  is  a  mapping 
of  the  Input  Plane  calculated  by  considering  the  eye  as  a  F-Tan  lens 
with  a  focal  length  of  unity.  In  all  output  mappings,  the  value  of  the 
focal  length  is  taken  as  unity,  allowing  mappings  for  a  lens  of  another 
focal  length  to  be  represented  by  multiplying  the  mapping  coordinates 
by  the  desired  focal  length. 

The  second  mapping  is  from  the  dome  surface  through  the  view  plane 
and  onto  the  target  plane.  The  mappings  produced  by  MAPTAO  are 
mappings  at  the  view  plane,  which  provide  a  perspective  view  of  the  ob¬ 
ject  points  on  the  dome  from  the  projection  point.  A  perspective  view 


of  the  target  plane  from  the  projection  point  (a  view  of  the  required 
raster  pattern)  may  be  obtained  by  a  rotation  of  the  output  mapping 
about  its  center  by  180°  and  observing  the  pattern  through  the  reverse 
side  of  the  mapping. 

The  coordinates  for  the  view  plane  output  mapping  can  also  be 
placed  in  table  form.  Each  table  contains  specific  information  on  the 
projection/viewing  system  and  the  type  of  mapping  used.  The  coor¬ 
dinates  are  printed  in  the  table  according  to  their  relative  position  on 
the  raster  plane.  Also  placed  in  the  table  output  are  the  coordinates  of 
the  Comparison  Plane,  allowing  distortion  calculations  to  be  made 
from  the  table.  Use  of  the  Comparison  Plane  coordinates  for  calcula¬ 
tions  requires  the  scaling  of  the  coordinates  to  reduce  the  Comparison 
Plane  height  to  the  height  of  the  View  Plane.  The  scaling  factor  is  defin¬ 
ed  as  the  ratio  of  the  View  Plane  height  to  the  height  of  the  Comparison 
Plane.  For  purposes  of  distortion  calculations,  the  GPE  is  then  found 
by  effectively  overlaying  the  View  Plane  on  the  Comparison  Plane  via 
the  coordinate  tables. 

Subroutine  TBLD1ST  uses  the  coordinates  sent  to  the  mapping  tables 
to  calculate  distortion  percentages  for  nine  points  in  the  field.  These 
nine  points,  assuming  an  odd  number  of  points  on  the  plane,  are  the 
four  corner  points,  the  central  point,  and  the  four  points  at  the  mid¬ 
point  of  each  edge  of  the  plane.  The  percentages  found  ore  included  on 
the  mapping  table  output. 

PROGRAM  OPERATION 
Running  the  Program 

In  order  to  operate  the  pregram,  variables  describing  the  projec¬ 
tion/viewing  system  must  he  entered.  Upon  instructing  the  computer  to 
execute  the  program  (RUN  MAPTAG),  the  user  is  prompted  to  enter 
the  necessary  values.  Definition  of  the  view  window  starts  with  the  loca¬ 
tion  of  the  center  of  the  view  window  (PCTR).  This  location  is  in  3-D 
coordinates  relative  to  the  screen  centei  and  must  be  on  the  screen  sur¬ 
face.  Next,  the  height  and  width  in  degrees  and  the  number  of  points 
across  and  down  the  window  are  entered.  The  maximum  height  or 
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width  of  any  window  is  less  than  1 80°  due  to  the  method  of  creating  the 
input  plane.  Additionally,  the  number  of  points  should  be  odd  for  use 
in  the  dir  jrtion  calculation  subroutine. 

Describing  the  rest  of  the  system  requires  entering  the  radius  of  the 
screen,  R,  the  projector  location  BX(FX,  FY,  FZ),  and  the  viewpoint, 
EN(XE,  YE,  ZE).  The  units  for  these  coordinates  are  arbitrary,  but 
must  be  the  same  for  all  3-D  coordinates.  As  an  option,  the  projector 
axis  can  he  offset  from  PCTR  during  operation  of  the  program  by 
entering  degree  offsets  other  than  zero  for  VOFFP  and  HOFFP. 

The  rest  of  the  values  to  be  entered  are  concerned  with  the  type  of 
output  mapping  and  table.  The  type  of  lens  mapping  is  chosen  by  enter¬ 
ing  a  value  from  1  to  4  corresponding  to  the  four  lens  mappings 
available.  If  a  Tan  9  mapping  with  a  primary  distortion  is  desired,  then 
the  user  is  prompted  to  enter  a  distortion  factor  (DFACTOR).  Addi¬ 
tional';  the  user  is  instructed  to  enter  a  graphic  scaling  factor 
(FSCALE),  screen  magnification  factor  (SMAG),  and  offsetting  values 
for  the  origin  of  the  screen  (XBIAS,  YBIAS)  in  inches.  Finally,  the  user 
can  decide  if  table  output  is  wanted  during  the  present  run  of  the  pro¬ 
gram. 

After  the  program  has  completed  the  graphics,  the  user  is  instructed 
to  enter  “C”  to  continue  execution.  The  user  is  then  offered  the  option 
of  changing  the  projection  axis  offset,  graphic  scaling,  screen 
magnification,  and  table  output  option.  If  a  change  is  desired,  the  pro¬ 
gram  again  prompts  the  user  to  enter  values  and  the  graphic  screen  is 
cleared  in  order  to  draw  a  new  perspective  view  according  to  the  new 
values.  If  no  changes  are  desired,  the  program  will  clear  the  graphic 
screen  and  execution  is  stopped. 


SAMPLE  PROJECTION  SYSTEMS 

As  examples,  the  results  of  two  sample  projection/viewer  systems  are 
included.  The  first  sample  is  a  simple  projector/viewer  arrangement 
where  the  viewer  is  placed  at  the  center  of  a  20  foot  radius  dome  and  the 
projector  is  located  at  the  3-D  location  (0.0,0.0,12.0)  inches.  This 
results  in  a  12  inch  displacement  of  the  projector  directly  above  the 
viewer.  In  this  case  the  projected  View  Window  subtends  angles  of 
70 “vertical  by  90“  hoiizontal  from  the  viewpoint.  On  this  View  Win¬ 
dow,  there  is  an  1 1  by  1 1  raster  pattern  which  forms  one  hundred  rec¬ 
tangular  blocks  as  depicted  on  figure  9.  The  window  is  centered  on  the 
surface  of  the  dome  at  the  3-D  location  (240.0,0.0,0  0)  inches,  or  at  the 
intersection  of  the  X-axis  with  the  dome  surface.  Table  1  shows  the  pro¬ 
gram  prompts  and  user  inputs  to  describe  the  above  system.  Figures  10, 
11,  and  12  show  the  required  raster  shapes  to  be  projected  from  the 
defined  projector  location  for  the  use  of  projection  lenses  with  mapping 
functions  of  Tan  9  ,  Sin  9  ,  end  9  ,  respectively. 

The  second  projection/viewer  arrangement  is  a  projector  arrange¬ 
ment  with  the  View  Window  and  projector  located  on  opposing  sides  of 
the  dome  surface.  In  this  system  the  viewpoint  is  again  coincident  with 
the  center  of  a  20  foot  radius  dome.  However,  the  projector  is  located 
behind  the  viewer  on  the  surface  of  the  dome  at  the  3-D  location 
(-218.0,0.0,101.0)  inches,  and  the  View  Window  is  placed  at  the  3-D 
location  (218.0,0.0,-101.0)  inches.  This  arrangement  directs  the  optical 
axis  of  the  projector  to  pass  through  the  center  of  the  dor  <e  on  its  way 
to  the  center  of  the  View  Window,  and  provides  a  for  a  symmetrical 
projection  onto  the  window  at  a  distance  of  twice  the  dome  radius,  or 
40  feet.  The  View  Window  contains  an  11  by  II  raster  pattern  and 
subtends  angles  of  160“  by  160“  from  the  viewpoint  as  shown  in  Figure 
13.  Figures  14,  13,  and  16,  with  tables  5, 6,  and  7,  describe  the  required 
raster  shapes  for  example  2. 

ANALYSIS  OF  THE  SAMPLE  SYSTEMS 

Analysis  of  the  required  raster  shapes  for  a  projection  system  in¬ 
volves  looking  at  the  graphic  mapping  outputs  and  their  corresponding 
table  outputs.  From  these  a  visualization  of  the  raster  shapes  is  ob¬ 
tained,  along  with  the  distortion  percentages  related  to  each  mapping. 

Examining  the  coordinate  tables  2,  3,  and  4,  which  are  for  the  first 
example,  specific  information  for  each  type  of  lens  in  this  projection  ar¬ 
rangement  is  revealed.  The  tables  are  labeled  as  to  the  type  of  lens  map¬ 


ping  function  considered  along  with  identification  of  the  projector, 
viewer,  and  view  window  locations.  The  tables  contain  the  coordinates 
of  the  intersection  points  for  the  distorted  raster  shape  and  the  coor¬ 
dinates  of  the  non-distorted  Comparison  Plane.  Notice  that  the  coor¬ 
dinates  are  precceded  by  the  labels  (X,  Y)  and  (XC,  YC)  which  denote 
the  coordinates  of  the  View  Plane  and  Comparison  Plane  in  turn.  These 
coordinates  are  Included  in  the  tables  in  the  same  manner  that  the  raster 
lines  are  drawn  on  the  View  Window,  that  is,  left  to  right  and  top  to 
bot'om.  The  tables  provide  only  the  coordinates  of  the  points  used  in 
the  distortion  calculation  routine,  and  not  all  the  points  of  the  1 1  by  1 1 
pattern.  The  coordinates  of  alt  the  points  are  available,  but  are  not  in¬ 
cluded  heie  due  >o  the  size  of  the  tables. 

The  first  output  mapping  for  example  1,  figure  9,  is  a  tangent  map¬ 
ping  of  the  View  Window,  or  equivalently,  the  Comparison  Plane.  This 
is  the  raster  shape  that  the  viewer  should  see  for  the  condition  of  no 
distortion.  The  tangent  mapping  for  this  projection  system,  figure  10, 
shows  a  maximum  distortion  of  2.6%,  with  symmetrical  distortion 
about  the  center  vertical  line  of  the  raster  pattern.  All  the  distortion 
percentages  for  this  mapping  are  low  and  reflect  the  close  proximity  to 
the  viewpoint  of  a  projector  with  a  F  Tan#  mapping  function.  Looking 
at  the  F-#  andF-Sin#  outputs,  the  maximum  distortion  "ercentages 
jump  to  2U.6V*  and  20.3V* .  These  percentaghes  are  la:g'  1  reflect  the 
departure  of  the  individual  lens  mapping  functions  •rv.-n  a  distor¬ 
tionless,  or  F-Tan  9  ,  lens.  In  this  system,  a  lens  with  a  i-Tan  9  map¬ 
ping  would  be  preferred  due  to  the  closeness  of  the  projector  and 
viewer,  and  the  less  than  90°  projection  angles  required. 

The  second  example  is  offered  as  an  extreme  case  compared  to  the 
preceeding  case.  The  separation  of  the  projector  and  viewer  is  increased 
to  20  feet  and  the  Vita  Window  is  required  to  fill  a  FOV  of  160  °  by 
160“  at  the  viewpoint.  Examining  the  mappings  and  tables  for  this 
second  system,  it  can  be  seen  that  the  distorted  raster  shapes  for  alt  the 
mapping  functions  fall  within  the  90°  reference  circle.  In  fact,  the  max¬ 
imum  projection  angle  for  any  of  the  lenses  is  approximately  80°.  This 
angle  is  appropriate  considering  the  View  Window  to  fill  a  160  by  160“ 
FOV  at  the  viewpoint  with  projection  from  a  distance  of  twice  the  dome 
radius.  A  projection  lens  w;th  the  TFOV  of  90“  would  provide  more 
than  enough  of  the  projection  field  required  to  fill  the  View  Window. 

The  distortion  percentages  for  all  three  lens  types  are  quite  large  and 
are  approximately  equal.  Examining  tables  3,  6,  and  7,  the  maximum 
distortion  required  for  the  F-  9  mapping  is  slightly  greater  than  the 
Tangent  mapping  (18.09V*  vs.  18.90V*),  while  the  F-Sin  9  mappingre- 
quires  the  most  raster  shaping  at  19.22V*.  These  raster  shapes  may  be 
hard  to  implement  by  raster  shaping  alone  as  can  be  seen  in  figures  14, 
13,  and  16. 

Other  information  to  be  gained  from  the  program  output  concerns 
the  redistribution  of  the  raster  points  on  the  projection  lens  target 
plane.  In  the  second  system,  the  points  are  crowded  together  near  the 
edges  of  the  target  plane  with  only  a  few  points  in  the  center  of  the 
target  plane.  The  effect  of  this  distribution  depends  on  the  projection 
lens  in  use,  but  would  tend  to  reduce  the  resolution  capacity  of  the  im¬ 
aging  system.  This  is  due  to  the  crowding  of  resolution  elements 
(Resets)  on  the  target  plane  where  most  lenses  lose  resolution  capability, 
and  lack  of  resets  where  most  lenses  have  their  greatest  resolution. 

SUMMARY 

The  orogram  MAPTAG  is  a  very  useful  tool  for  determining  the 
required  raster  shape  to  be  projected  which  provides  for  distortionless 
viewing.  The  graphic  output  provides  visualization  of  the  distortions 
encountered  in  dome  displays.  The  table  allows  distortion  calculations 
to  be  performed  and  can  also  provide  for  equations  that  describe  the 
raster  distortion,  line  by  line. 

The  subroutines  utilized  by  MAPTAG  have  been  written  in  a  general 
form  to  allow  their  use  in  building  other  specific  distortion  routines. 
The  flowcharts  and  program  coding  have  been  documented  and  are 
available  to  the  general  public.  (9)  Extension  of  the  program,  to  provide 
more  information  about  projection  systems,  is  being  investigated. 
These  programs  provide  an  excellent  basis  for  distortion  analysis  of 
video  projection  systems  and  efforts  are  being  taken  to  include  projec¬ 
tion  of  generalized  imagery. 
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EXAMPLE  1 

BUNMAPTAQ 

de  sense  planar  input  screen 

ENTER  0E8FED  COORDINATES  FOR  PLACEMENT  OF  CENTER 
OP  PLANE  W  8PHERE  COOROWATE  SYSTEM ,  THE  X-AXIS  B  POSITIVE 
FORWARD  WITH  THE  Y-AXIS  POSITIVE  TO  THE  LEFT,  AND  THE  Z-AXJS 
IS  POSITIVE  UPWARDS 
240,0,0 

ENTER  VERTICAL  AND  HCWZONTAL  FOV'S  IN  DEGREES 
70,00 


TAN  THETA  MAPPWQ  OOORDWATES 
THE  VBWPOWT  IS  AT  : 

X  -  0.0000  Y  -  0.000  Z  -  0.000 

OBSERVING  A  UNEAR  RASTER  PATTERN  ON  THE  8CREEN 
FILLING  a  FIELD  OF  70.000  DEGREES  VERTICALLY,  AND 
00.000  DEGREES  HORIZONTALLY  ABOUT  A  POINT  X« 
240.000  Y-  0.000  Z-  0.000  DEFINED  AS 

CENTER  OF  "FOV". 

THE  PROJECTOR  IS  AT: 

X  •  0.0000  Y  -  0.000  Z  -  12.000 

THE  RADIUS  OF  THE  8PHERICAL  SCREEN  IS  240.000 


ENTER  NUMBER  OP  POWTS  ALONG  VERTICAL  AND  HORIZONTAL 

X 

1.033 

0.000 

•1.033 

AXES 

Y 

0083 

0.712 

0.883 

11,11 

XC 

1.000 

0.000 

•1.000 

DESCRIBE  SPHERK2AL  SCREEN  DISPLAY  SYSTEM 

YC 

0.700 

0.700 

0.700 

ENTER  8PHERCAL  SCREEN  RADIUS 

X 

0.080 

0.000 

•0.888 

240 

Y 

■0.021 

0.000 

•0.021 

ENTER  PROJECTOR  PO8ITI0N  (FX.FY.FZ) 

0,0,12 

XC 

1.000 

0.000 

-1.000 

ENTER  VIEWPORT  POSITION  (XE.YE.ZE) 

0.0.0 

YC 

0.000 

0.000 

0.000 

ENTER  VERTICAL  AND  HORIZONTAL  OFFSET  FOR  PROJECTION 

X 

0.804 

0.000 

-0.884 

DETECTION  (DEGREES) 

0.0 

ENTER  SCALE  FACTOR  .  SCREEN  MAS  .  YBIAS.  XBIAS  NORMALLY 

Y 

•0.702 

-0.086 

-0.702 

XC 

1.000 

0.000 

-1.000 

SCREEN  MAG  ■  1,  XBIAS  ■  0.  YBAS  ■  0,  THE  SCALE  FACTOR 
DETERMWES  THE  TFOV  (FSCALE  •  80) 

XC 

•0.700 

-0.700 

•0.700 

1.6, 1.1, 0.0 

DISTORTION  PERCENTAGES 

PICK  WHCH  TYPE  OF  MAPPNG  D£ SPIED.  FOR  A  TANGENT  MAPPING 

ENTER  "1",  FOR  AN  IDEAL  R-THEATA  MAPPNG  ENTER  "2",  FOR 

THE  TARGET  PLANE  HEIGHT  IS 

1.387 

TANTHETA  MAPPNG  WITH  PRIMARY  DISTORTION  ,  ENTER  “3  ", 
FOR  SN  THETA  MAPPWG,  ENTER  "4" 

1 

2 

3 

1 

2.44% 

0.84% 

2.44% 

F  TABLE  OUTPUT  IS  DE SPIED  TYPE  TRUE  ;  IF  NOT  FALSE 

4 

6 

6 

T 

1.48% 

0.00% 

1.48% 

7 

B 

8 

TYPE  C  TO  CONTWUE  EXECUTION 

1 

2.00% 

1.08% 

2.80% 

SC 

TO  CHANGE  VOFF.HOFF, MAG, FSCALE, LIABLE, TYPE  OF  MAPPING 

ENTER  "T",  IF  NOT  ENTER  "F" 

TABLE  2 

p 

TO  CHANGE  SCREEN  MAGNIFICATION. ENTER  T  ,TO  STOP  ENTER  F 

VIEW  WINDOW 

F 

70*  V  x  SO* 

H 

TANGENT  MAPPING 


Figure  0. 


24 


B- THETA  MAPPING  COORDHATE8 


X 


THE  PROJECTOR  »  AT  : 

■  0.0000  Y  -  0.000  Z 


12.000 


THE  VIEWPOINT  IS  AT  :  ' 

X  -  0.0000  Y  -  0.000  Z  -  0.000 

OBSERVING  A  LINEAR  RASTER  PATTERN  ON  THE  SCREEN 
FILLING  A  FIELD  OF  70.000  DECREES  VERTICALLY,  ANO 
90.000  DEGREES  HORIZONTALLY  ABOUT  A 
X-  240.000  Y-  0.000  Z  -  0.000  DEFINED  AS  < 

OF  "FOV". 

THE  PROJECTOR  IS  AT  : 

X  -  0.0000  Y  n  0.000  Z  “  12.000 

THE  RAOfcJS  OF  THE  SPHERICAL  SCREEN  IS  240.000 


X 

0.742 

0.000 

•0.742 

Y 

0.498 

0.018 

0.498 

XC 

1.000 

0.000 

•1.000 

YC 

0,700 

0.900 

0.700 

X 

0.784 

0.000 

-0.784 

Y 

■0.015 

0.000 

•0.016 

XC 

1.000 

0.000 

-1.000 

YC 

0.000 

0.000 

0.000 

X 

0.706 

0.000 

-0.706 

Y 

-0.614 

-0.601 

-0.614 

XC 

1.000 

1.000 

•1.000 

YC 

•0.700 

-0.700 

DISTORTION  PERCENTAGES 

THE  TARGET  PLANE  HEIGHT  IS 

1.219 

1 

2 

3 

26.87%  6.69% 

26.87% 

4 

o 

8 

17.76%  0.00% 

17.75% 

7 

8 

g 

28.68%  8.16% 

28.58% 

TABLE  3 

VIEW  WINDOW 

70" 

X 

• 

8 

X 

> 

R-THETA  MAPPING 


Figure  1 1 . 


SIN  THETA  MAPPfJQ  COORDINATES 

THE  VIEWPOINT  IS  AT  ; 

-  0.000  Y  -  0.000  Z  -  0.000 

OBSERVING  A  LINEAR  RASTER  PATTERN  ON  THE  SCREEN  FILLING 
A  FIELD  OF  70.000  DEGREES  VERTICALLY,  AND  90.000 
DEGREES  HORIZONTALLY  ABOUT  A  POINT 
X-  240.000  Y-  0.000  Z>  0.000 
DeFN£D  AS  CENTER  OF  "FOV". 


THE  RADIUS  OF  THE  SPHERICAL  SCREEN  IS  240.000 

X 

0.647 

0.000 

•0.6 

Y 

0.434 

0.580 

0.4 

XC 

1.000 

0.000 

-1.0 

YC 

0.700 

0.700 

0.7 

X 

0.706 

0.000 

-0.7 

Y 

•0.015 

0.000 

•0.0 

XC 

1.000 

0.000 

•1.0 

YC 

0.000 

0.000 

0.0 

X 

0.819 

0.000 

-0.6 

Y 

-0.451 

•0.565 

-0.4 

XC 

1.000 

0.000 

-1.0 

YC 

•0.700 

•0.700 

•0.7 

DISTORTION  PERCENTAGES 

THE  TARGET  PLANE  HEIGHT  IS  1 . 1 45 

1  2 

3 

19.17% 

0.04%  10.17% 

4  6 

6 

9.82% 

0.00%  9.82% 

7  8 

9 

20.32% 

0.84%  20.32% 

TABLE  4 

VIEW  WINDOW 
70"  V  x  90*  H 


TANGENT  MAPPING 
Flour#  13. 


TAN  THETA  MAPPING  COORDINATES 
THE  VIEWPOINT  »  AT  : 

X  -  0,0000  Y  -  0.000  Z  -  0.000 

OBSERVING  A  LINEAR  RASTER  PATTERN  ON  THESCREEN 
FILLING  A  FIELD  OF  160.000  DEGREES  VERTICALLY. 
AND  160.000  DEGREES  HORIZONTALLY  ABOUT  A  POINT 


X-  216.000  Y- 
AS  CENTER  OF  "FOV". 

0.000  Z- 

-101.000  DEFINED 

THE  PROJECTOR  ®  AT  : 

X 

-  -218.0000  Y  - 

0.000  Z  - 

101.000 

THE  RADIUS  OF  THE  SPHERICAL  SCREEN  IS 

240.000 

X 

0.624 

0.000 

-0.624 

Y 

0.624 

0.838 

0.624 

XC 

8.671 

0.000 

•6.671 

YC 

5.671 

5.671 

5.671 

X 

0.836 

0000 

-0.838 

Y 

0000 

0.000 

0.000 

XC 

6.671 

0  000 

-5.671 

YC 

0.000 

0.000 

0.000 

X 

0.624 

0.000 

-0.624 

Y 

-0.624 

•0.838 

-0.624 

XC 

5.671 

0.000 

•5.671 

YC 

•5.671 

•6671 

•5.671 

DISTORTION  PERCENTAGES 


1 

18.00% 

2 

0.00% 

3 

18.00% 

4 

0.00% 

5 

0.00% 

6 

0.00% 

7 

18.00% 

8 

0.00% 

0 

18.00% 

TABLES 

110*  FOV 


TANGENT  MAPPING 


R-THBTA  MAPP WG  OOORBNATE8 
THE  VIEWPORT  SAT; 


X  - 

0.000  Y  - 

0.0000  z  • 

0.000 

OBSERVING  A  UNBAR  RASTER  PATTERN  ON  THE  SCREEN  FILL- 

ING 

A  FIELD  OF 

160.000  DEGREES  VERTICALLY, 

AND 

160.000  DEGREES  HORIZONTALLY  ABOUT  A  POINT 

X- 

218.000  Y. 

0.000  Z- 

-101.000  DEFINED  AS 

CENTER  OF  "FOV". 

THE  PROJECTOR  IS  AT  : 

X  -  ■ 

218.0000  Y  - 

0.000  Z  • 

101.000 

THE  RADIUS  OF  THE  8PHERICAL  SCREEN  18 

240.000 

X 

0.611 

0.000 

-0.611 

Y 

0.611 

0.608 

0.611 

XC 

6.671 

0.000 

-6.671 

YC 

6.671 

6.671 

6.671 

X 

0.606 

0.000 

-0.608 

Y 

0.000 

0.000 

0.000 

XC 

5.671 

0.000 

-6.671 

YC 

0.000 

0.000 

0.000 

X 

0.611 

0.000 

•0.611 

Y 

-0.61 1 

-0.608 

-0.611 

XC 

6.671 

0.000 

-5.671 

YC 

-5.671 

-5.671 

•5.671 

DISTORTION  PERCENTAGES 


1  2  3 


16.00% 

0.00% 

18.00% 

4 

5  6 

0.00% 

0.00% 

0.00% 

7 

8  0 

18.00% 

0.00% 

18.00% 

TABLES 


110*  FOV 


R-THETA  MAPPING 


Figure  IS. 


1 


9*1  THETA  MAPPING  OOOACXNATI8 
THE  VIEWPOINT  IS  AT: 

X  -  0.0000  Y  •  0.000  Z  -  0.000 

OBSERVNG  A  UNEAR  RASTER  PATTERN  ON  THE  SCREEN  P1UJNQ 
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160.000  OEGREES  HORIZONTALLY  ABOUT  A  POINT 
X.  218.000  Y «  0.000  Z-  -101.000  DEFINED  A8 

CENTER  OF  "FOV". 

THE  PROJECTOR  18  AT  : 

X  -  -216.0000  Y  •  0,000  Z  -  101.000 
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ABSTRACT 

Large  dome  simulators  for  air-to-air  and  air-to-ground  aircraft  weapons  tactics  trainers  are 
coming  Into  prominence.  In  such  devices,  the  training  objective  Is  to  Improve  pilot  proficiency 
and  coordination  by  allowing  the  pilot  to  train  In  the  use  of  weapons  In  realistic  operational 
and  threat  environments.  Typically,  the  pilot  will  fight  against  a  TV  projected  Image  that  Is 
slewed  across  the  field  of  view.  Sometimes  neither  the  pilot's  eyes  nor  the  TV  projector  lens  Is 
located  »t  the  center  of  the  dome  screen.  From  geometrical  considerations,  a  standard 
rectangular  or  square  TV  Image  looks  distorted  to  the  pilot.  The  degree  and  shape  of  the 
distortion  changes  with  the  shift  In  location  of  the  "target"  on  the  screen  relative  to  the  pilot 
as  It  Is  slewed  either  by  servo  pointing  the  entire  projector  or  by  optical  means. 

This  paper  describes  the  design  and  development  of  a  TV  projector  that  Is  capable  of 
predistorting  the  TV  raster  such  that  from  the  pilot's  viewpoint  the  Image  will  look  rectilinear 
at  all  times.  This  dynamic  raster  shaping  can  be  updated  at  the  TV  field  rate  (typically 
60  times  per  second)  so  that  there  Is  no  perceptible  jumpiness  in  the  Image  as  the  shape  Is 
varied.  In  addition,  the  raster  can  be  zoomed  to  create  the  appearance  of  distance  change  to  the 
target  and  rotated  to  compensate  the  effects  caused  by  the  mirror  steering  of  the  target  Image 
across  the  dome. 


INTRODUCTION 

The  layout  of  a  typical  aircraft  weapons 
tactics  trainer  such  as  presently  being  developed 
for  the  U.S.  Navy  as  Training  Device  2E7  Is  shown 
in  Figure  1.  Each  of  two  40-ft  diameter  domes 
houses  a  simulated  cockpit  of  a  high  performance 
jet  fighter  a1rcraft--an  F/A-18  In  the  case  of 
Device  2E7.  The  Images  from  three  background  TV 
projectors  and  four  target  TV  projectors  are 
projected  on  the  Inside  of  the  dome  to  present  an 
all-encompassing  dynamic  scene  to  the  pilot.  The 
images  are  generated  by  digital  techniques  and  the 
dynamics  of  the  system  are  controlled  by  a  large 
central  computer.  With  this  arrangement  each  of 
two  pilots  can  either  engage  In  simulated  combat 
with  each  other  or  each  can  engage  simultaneously 
In  battle  against  the  computer. 

Each  of  the  three  background  projectors  Is 
dedicated  to  a  120-degree  segment  of  the  dome  and 
presents  the  Image  of  the  earth  and  sky  environment 
surrounding  the  aircraft.  Each  of  the  four  target 
projectors  provides  means  to  slew  the  Image  of  a 
target  or  friendly  aircraft  across  the  field  of 
view  of  the  pilot.  Since  neither  the  pilot's  eye 
point  nor  the  target  projectors  are  necessarily 
located  on  the  geometrical  center  of  the  dome, 
standard  rectilinear  TV  Images  may  look  distorted 
to  the  pilot.  The  degree  and  shape  of  the 
distortion  changes  with  the  location  of  the 
projected  target  on  the  dome  screen  relative  to  the 
pilot  as  it  Is  slewed  across  his  field  of  view. 

DYNAMIC  RASTER  SHAPING 

Systems  Research  Laboratories,  Inc.  (SRL)  has 
provided  TV  projectors  for  simulation  and  training 
where  either  the  whole  projector  head  Is 
mechanically  slewed  under  servomechanism  control 
(as  shown  In  Figure  2)  or  the  projector  head 
remains  stationary  and  the  Image  Is  slewed  across 
the  domed  screen  by  use  of  articulated  optics  (as 
shown  In  Figure  3a  and  3b).  In  either  case, 
computer  controlled  servomechanisms  are  used  to 


keep  the  images  in  focus  as  the  distance  to  the 
screen  changes  with  pointing  angle.  The  glmballng 
of  the  whole  projector  head  is  better  when  using 
wide  fleld-of-view  lens,  while  the  articulated 
lens  Is  more  practical  with  narrow  fleld-of-view 
lens. 

We  are  familiar  with  how  a  slide  projector 
Image  becomes  distorted  If  the  slide  projector  Is 
not  placed  squarely  to  the  screen.  If  It  Is  too 
low,  the  Image  Is  distorted  such  that  the  Image  of 
a  square  looks  more  like  a  trapezoid  with  a  larger 
top  than  bottom.  We  also  know  that  a  projected 
picture  that  Is  square  on  the  screen  can  look 
distorted  when  viewed  off  axis.  These  effects  are 
compounded  In  the  case  of  the  target  projectors 
where  the  Image  on  the  TV  projection  cathode  ray 
tube  Is  flat  end  rectilinear  and  the  screen  Is 
concave  and  off-axis  to  the  projected  Image.  The 
answer  to  the  problem  created  by  such  geometrical 
distortions  Is  to  predistort  the  TV  Image  before  It 
is  projected  such  that  It  looks  correct  to  the 
viewer. 

Several  approaches  to  predistorting  the 
Images  can  be  taken.  The  digital  Image  generator 
can  be  programmed  to  predistort  the  video.  This 
requires  a  large  Increase  In  computing  power. 
Likewise,  a  double  ended,  optically  coupled  scan 
converter  such  as  the  SRL  Model  342A  (effectively 
a  TV  camera  viewing  rectilinear  video  displayed  on 
a  predistorted  CRT  raster;  therefore,  generating  a 
predistorted  video  at  the  TV  camera  output,  can  be 
used.  This  method  has  a  built-in  time  delay  of 
one  field  time  and  requires  extremely  good  (but 
achievable)  electro-optics  In  order  not  to  degrade 
resolution.  The  method  described  In  this  paper 
uses  an  approach  where  rectilinear  video  Is 
displayed  on  a  CRT  projector  which  provides 
dynamic  raster  shaping. 

The  development  of  the  predistortion 
algorithm,  though  straightforward,  Is  still 
moderately  complex.  Fortunately,  the  pre¬ 
distortion  can  be  Implemented  In  a  practical 


system  by  modifying  the  TV  horizontal  and  vertical 
sweep  signals  with  a  series  of  linear  and  nonlinear 
mathematical  expressions  In  the  analog  domain.  The 
functions  that  need  to  be  Implemented  are: 

position 

size 

linearity 

trapezoidal 

plncushlon/barrel 

curvature 

rotation 

orthogonality 

The  effect  of  these  functions  on  a  rectilinear 
raster  Is  shown  In  Figure  4.  All  terms  except  for 
rotation  can  be  solved  using  linear  analog 
computational  techniques.  Rotation  requires 
sine/cosine  coordinate  transformation. 

In  the  case  of  the  Navy's  F/A-18  weapons 
tactics  trainer  Device  2E7,  the  central  computer, 
either  from  canned  programs  or  from  the  responses 
of  the  two  adversary  pilots,  keeps  track  of  the 
position  of  the  target  aircraft  and  generates  the 
pointing  angles  of  the  respective  target 
projectors.  This  Information  Is  translated  to 
servomechanism  Information  which  Is  transmitted  to 
the  projector  focus  servo  and  to  the  articulated 
lens  azimuth  and  elevation  mirrors.  It  also 
generates  at  a  TV  field  rate  (60  flelds/second)  the 
coefficients  for  all  the  linear  predistortion 
functions  and  both  the  sine  and  cosine  of  the 
rotation  function  In  a  digital  format.  Within  the 
control  unit  of  the  TV  projector  electronics,  this 
digital  data  Is  converted  to  analog  and  the 
remainder  of  the  computation  Is  accomplished  In  the 
analog  domain.  Figure  4  Is  a  series  of  drawings 
showing  the  components  of  the  various  raster 
distortion  terms  that  can  be  applied  to  the  sweep 
circuits.  These  predistortion  terms  are  shown  only 
for  the  horizontal  axis  for  ease  In  visualizing  the 
effect.  In  actuality,  a  portion  of  each  of  these 
corrections  in  both  the  horizontal  and  vertical 
axis  as  well  as  rotation  would  be  present,  creating 
a  very  complex  raster  such  as  shown  In  Figure  5. 

OTHER  CONSIDERATIONS 

Dynamic  raster  shaping  places  several  other 
major  requirements  upon  the  projector.  Normal  TV 
displays  and  projectors  use  resonant  horizontal 
deflection  amplifiers  due  to  their  power  conserving 
characteristics  and  due  to  their  relative 
simplicity.  Though  some  raster  correction  can  be 
accomplished  In  resonant  deflection  amplifiers, 
only  those  corrections  which  are  symmetrical  about 
the  center  of  sweep  can  be  Implemented  simply. 
This  eliminates  curvature  correction  which  Is 
needed  anytime  the  projector’s  optical  axis  Is  not 
collnear  with  a  line  drawn  from  the  center  of  the 
dome  to  the  dome  1tself--a  situation  that  occurs  at 
only  one  pointing  angle  with  each  projector.  Also, 
raster  rotation  Is  not  possible  since  the  rapid  TV 
retrace  can  be  made  In  only  one  direction  with 
resonant  circuits.  For  continuous  raster  rotation, 
both  the  vertical  and  horizontal  amplifiers  must  be 
equally  as  fast  In  both  directions  and  as  fast  as 
each  other.  With  these  restraints  on  the  system, 
It  Is  necessary  to  use  Identical  linear  current 
feedback  amplifiers  for  both  the  horizontal  and 
vertical  deflection  amplifiers. 


To  conserve  power,  an  SRL  patented  smooth 
actuating  voltage  boost  Is  used  whenever  the 
deflection  amplifiers  sense  It  Is  In  a  flyback 
mode--1ndependent  of  direction  of  flyback.  (1)  In 
practice,  In  each  dome  the  four  projectors  are 
used  In  two  pairs  allowing  the  projection  of  two 
Independent  targets  In  order  that  one  projector 
can  take  over  from  the  other  when  It  Is  slewed  to 
a  position  such  that  the  Image  would  be  obstructed 
by  the  simulated  aircraft  fuselage.  Again,  In 
order  to  conserve  power,  which  would  require  an 
additional  load  on  the  dome's  air  conditioning 
system,  the  SRL  projectors  have  a  deflection 
amplifier  powerdown  feature  which  Is  used  whenever 
the  projector  Is  not  selected  by  the  computer. 

Up  to  250:1  raster  zooming  Is  also  available 
In  the  projector  to  create  the  effect  of  variable 
distance  to  the  target.  However,  small  rasters  on 
a  TV  projector  operating  at  up  to  60  watts  screen 
dissipation  are  potential  problems  due  to  phosphor 
burning.  The  SRL  projector  calculates  the  actual 
raster  area  and  limits  the  video  drive  to  a  safe 
short-term  value.  This  allows  the  display  of 
bright  objects  for  short  periods  of  time,  but 
long-term  protection  Is  up  to  the  central 
computer. 

CONCLUSIONS 

Dome  simulators  are  proving  to  be  effective 
training  aids.  Steerable  TV  projectors  are  also 
proving  to  be  practical  means  of  providing  the 
Image  of  the  target.  As  more  realism  and 
precision  are  required  to  keep  airmen  proficient 
In  weapon  tactics,  more  emphasis  needs  to  be  given 
to  Image  fidelity.  This  requires  not  only  higher 
resolution,  but  higher  geometrical  fidelity.  The 
concept  of  providing  dynamic  raster  shaping  In  the 
TV  projector  instead  of  In  the  digital  Image 
generator  or  an  Intermediary  black  box  Is  both 
technically  superior  and  cost  effective  and  should 
find  Its  place  In  more  advanced  visual  systems.  I 
visualize  Its  use  spreading  to  air-to-ground  as 
well  as  present  air-to-air  weapons  tactics 
trainers. 
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Figure  3a.  Projector  Head  (without 


optica)  Used  with  Device  2E7  Target  Projector 
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Figure  3b.  Outline  Drawing  of  Projector  Head  Showing  Typical  Steering  Optics 
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Figure  A.  Photograph  of  Predistorted  Rectilinear  Raster 


Figure  5.  Typical  Raster 
A  -  Uncorrected 

B  -  With  Dynamic  Raster  Shaping 
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ABSTRACT 

A  study  on  the  effects  of  target  resolution  and  contrast  In  air  combat  engagements  was 
conducted  to  determine  the  potential  Impact  of  visual  display  characteristics  on  the 
effectiveness  of  air  combat  simulation  training.  A  large-scale  computer  model  of  air  combat 
engagements  was  used  to  Investigate  the  effects  of  2  and  4  arc  min.  of  target  resolution  and 
target/background  contrast  ratios  of  O.S  and  9.0.  The  study  results  are  discussed  In  the 
context  of  the  benefits  of  enhanced  visual  display  characteristics  In  practicing  the  skills 
required  In  air  combat. 


INTRODUCTION 

The  requirement  for  a  high  degree  of  realism 
In  the  generation  of  visual  scenes  in  the  dis¬ 
plays  of  manned  flight  simulators  has  been  a 
major  factor  In  Increasing  the  unit  cost  of 
simulators  used  In  aircrew  training.  The  train¬ 
ing  benefits  that  result  from  each  successive 
advance  In  visual  display  technology  must  be 
weighed  against  the  cost  Increase  associated 
with  that  technology  If  simulation  Is  to  remain 
a  viable  option  for  aircrew  training.  Agencies 
tasked  with  the  procurement  of  aircrew  training 
systems  are  repeatedly  tasked  with  the  problem 
of  cost-benefit  trade-offs  but  have  little  or  no 
objective  means  by  which  to  assess  the  extent  to 
which  training  will  be  affected  by  a  given 
Inprovement  In  a  device's  fidelity  with  actual 
aircraft  operations. 

An  area  where  flight  simulation  Is  likely  to 
play  an  Increasing  role  In  training  Is  In  air- 
to-air  combat.  Success  In  air-to-air  combat 
depends  on  the  skilled  execution  of  basic 
fighter  maneuvers  and  on  the  awareness  and 
understanding  of  the  tactical  situation. 

Simulation  training  has,  to  date,  stressed 
basic  fighter  maneuvering,  that  Is,  Instruction 
In  the  limits  of  an  aircraft's  flight  envelope, 
weapons  envelope  recognition  and  pre-planned 
offensive  and  defensive  fighter  maneuvers.  The 
Air  Force's  Simulator  for  Air-to-AIr  Combat 
(SAAC)  Is  an  example  of  this  kind  of  simulation 
training  for  air  combat.  However,  the  type  of 
training  that  Is  likely  to  have  the  greatest 
benefit  for  real-worlo'  combat  success  Involves 
flight  vs  flight  force-level  engagements  where 
many  aircraft  are  simultaneously  Involved  In  the 
same  gaming  area.  Only  exercises  Involving 
large  numbers  of  operational  aircraft  can 
currently  provide  realistic  training  In  these 
force-level  combat  environments.  The  costs  of 
such  exercises  In  fuel  consumed,  airframe  and 
engine  wear,  and  potential  for  aircraft  and 
aircrew  losses  make  It  difficult  to  conduct  this 
training.  The  alternative  to  providing  training 
In  combat  environments  likely  to  be  encountered 
by  pilots  is  through  ground-based  simulators. 
In  addition  to  reducing  costs,  simulation  would 
provide  virtually  unlimited  flexibility  In 
preparing  pilots  for  performance  with  and 


against  different  numbers  and  types  of  aircraft 
and,  therefore,  permit  training  In  key  areas 
such  as  situation  awareness,  coomunlcatlon  and 
control,  and  energy  management.  However,  simu¬ 
lation  technology  has  not  achieved  the  state 
where  visual  scene  generation  of  this  type  of 
environment  can  provide  the  necessary  degree  of 
realism  within  the  practical  limits  of  cost. 
For  example,  the  brightness  requirements  for 
presenting  a  full  field  oi  view,  daylight  scene 
is  1000  ft-L  or  greater  but  dome  visual  displays 
such  as  the  Navy's  VTRS  and  F-14  Hide  Angle 
Visual  Systems  (WAVS)  provide  a  brightness  of 
only  about  4  ft-L  and  0.4  ft-L,  respectively. 

Display  resolution  limits  are  of  even 
greater  concern  since  even  a  realistic  bright¬ 
ness  level  will  be  of  little  value  If  the  dis¬ 
play  resolution  is  not  sufficient  to  provide 
target  acquisition  and  identification  of  target 
attitude  changes  at  realistic  ranges.  Ideally, 
visual  perception  of  targets  In  simulators 
should  be  limited  only  by  the  optics  of  eye  and 
not  by  the  simulator  display.  Unfortunately, 
the  optimal  resolution  of  current  displays  is  In 
the  range  of  6-8  arc  min.  compared  to  1-2  arc 
min  resolution  capacity  of  the  eye.  Producing 
displays  with  .very  high  resolution  is  techni¬ 
cally  feasible")  but  the  cost  associated  with 
very  high  resolution  systems  Is  substantial. 
More  Important  from  the  viewpoint  of  training 
management  is  the  degree  to  which  training 
effectiveness  of  the  simulator  Is  Improved  by 
advances  in  display  quality.  In  the  majority  of 
cases,  the  real  training  value  of  a  simulator  Is 
only  known  well  after  the  device  Is  procured  and 
an  operational  test  conducted.  Design  specifi¬ 
cations  for  successful  training  of  certain  tasks 
can,  of  course,  be  done  to  a  limited  extent  on 
research  devices.  But  even  these  devices  are 
constrained  by  available  technology  as  well  as 
the  considerable  difficulty  In  conducting  con¬ 
trolled  transfer-of-trainlng  studies  to  deter¬ 
mine  the  cost-effectiveness  of  even  one  of  a 
variety  of  display  Improvements. 

This  paper  will  describe  an  alternative 
method  for  determining  the  potential  impact  of 
visual  display  characteristics  on  air  combat 
simulation  training  effectiveness  as  well  as 
presenting  Initial  data  from  this  ongoing 
research.  Our  objective  Is  to  provide  reliable 
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data  on  the  kinds  of  training  that  will  be 
possible  under  specific  visual  display  condi¬ 
tions  for  a  tactical  air  combat  simulator 
training  system.  For  the  sake  of  simplicity  and 
brevity  we  have  focused  on  resolution  and  con¬ 
trast  characteristics.  Investigations  Into 
other  display  characteristics  such  as  fleld-of- 
vlew/area-  of-lnterest  are  In  the  planning  stage. 

METHOO 

The  method  employed  in  this  study  had  to 
meet  the  requirements  of  any  useful  applied 
research,  l.e.,  control  of  the  variables  of 
Interest  and  valid  general Izabl llty  of  the 
results  to  the  operational  environment.  Addi¬ 
tional  constraints  of  time  to  conduct  the  needed 
studies  also  vere  considered.  Data  must  be 
available  at  the  time  In  which  decisions  on  a 
systems  training  value  are  being  made  If  such 
data  are  to  have  any  Impact  on  design  specifi¬ 
cations. 

The  advances  in  the  field  of  computer 
simulation  modeling  In  the  last  two  decades  are 
of  such  a  scope  that  the  utility  of  using  such, 
models  In  addressing  manned  flight  simulator 

design  Issues  deserved  Investigation.  A  com¬ 
puter  model  as  a  research  tool  has  the  potential 
to  meet  all  of  the  criteria  previously  mentioned. 

TAC  BRAWLER 

A  key  element  In  utilizing  a  computer  model 
to  Investigate  the  general  problem  of  simulation 
fidelity  and  the  specific  problem  of  visual 
display  requirements  Is  that  the  model 
accurately  depict  the  human  component  of  the 
task  at  hand.  Few  tasks  In  the  flight  regime 
depend  more  on  pilot  perception  and  decision¬ 
making  than  air-to-air  combat.  The  engagement 

model  that  was  ultimately  chosen  met  the  need  to 
simulate  this  human  component  of  realistic 
force-level  air  combat  engagements.  The  model 
chosen  for  this  research,  TAC  BRAWLER,  Is  a 

cooperative  air  combat  engagement  ,  model 
developed  under  the  auspices  of  the  Assistant 
Chief  of  Staff,  Studies  and  Analyses,  HQ  USAF. 
BRAWLER  was  designed  to  model  flight  vs  flight 
engagements  for  a  variety  of  fighter  aircraft 
types  and  weapons  systems.  Continual  inter¬ 
action  with  experienced  air  combat  pilots  In  the 
model's  development  assured  that  the  simulated 
pilot  responses  In  each  of  the  aircraft  involved 
In  the  engagement  accurately  depicted  skilled, 
but  not  perfect,  pilot  combat  performance. 
Further  validation  of  BRAWLER  has  been  carried 
out  In  conjunction  with  AIMVAL/ACEVAL  exercise. 

A  detailed  description  of  BRAWLER  Is  beyond 
the  scope  of  this  paper.  However,  a  brief  dis¬ 
cussion  of  the  model  architecture  of  8RAWLER 
(Figure  1)  emphasizing  the  visual  modeling  will 
aid  In  understanding  the  procedures  and  results 
that  follow.  The  architecture  of  BRAWLER  Is 
divided  Into  two  major  Information  arrays,  a 
Central  Status  Array  which  describes  the 
physical  parameters  of  the  engagement  and  a 
Mental  Status  Array  which  describes  each  pilot's 
perception  of  the  engagement.  The  Central 
Status  Array  maintains  Information  on  each  air¬ 
craft's  position  and  velocity,  missile  launches, 
etc.  for  all  aircraft  In  the  engagement.  The 


Mental  Status  Array  maintains  only  the  Infor¬ 
mation  available  to  a  pilot  based  upon  that 
pilot's  (Imperfect)  knowledge  of  the  situation. 
A  pilot's  awareness  of  the  actions  of  other  air¬ 
craft  in  the  engagement  is  based  upon  informa¬ 
tion  available  visually  (either  directly  or  by 
radar)  and  Information  from  flight  members  or 
ground  control.  The  decisions  trade  by  each 
pilot  In  the  engagement  are,  therefore,  based  on 
Information  that  would  be  available  to  that 
pilot  In  a  real-life  engagement.  This  approach 
to  the  modeling  of  pilot  behavior  In  BRAWLER 
allows  for  the  simulation  of  air  combat  skill 
expected  of  experience,  but  not  perfect,  pilots. 

Pilot  Visual  Perception. 

Since  this  re'port  addresses  the  Issue  of 
visual  display  effects  In  a  manned  flight  simu¬ 
lator,  a  detailed  descrlptloi  of  how  BRAWLER 
models  the  visual  perception  of  pilots  In  air 
combat  engagements  Is  necessary.  The  visual 
processes  of  the  simulated  pilot  In  BRAWLER  were 
systematically  varied  to  emulate  the  effects  of 
differing  display  characteristics  of  resolution 
and  contrast.  Target  inherent  contrast  values 
of  0.5  and  9.0  were  used  In  this  study  where 
inherent  contrast  Is  computed  by  the  formula: 

C  =  |Lt-Lb|/Lb 

where:  Lb  *  target  luminance, 

Lb  =  background  luminance. 

The  effect  of  target  contrast  on  the  likelihood 
of  detection  depends  on  the  position  of  the 
target  in  the  pilot's  visual  field.  As  a  conse¬ 
quence,  BRAWLER  models  the  effect  of  target  con¬ 
trast  utilizing  a  visual  detection  lobe.W 
The  lobe  delineates  the  threshold  contrast 
values  that  a  target  must  meet  In  order  to  be 
detectable.  The  equation  used  to  describe  the 
detection  lobe  for  foveal  vision  Is: 

Ct  =  1.55  +  15.2/B2,  (0  <  0.8  deg) 

The  equation  for  parafoveal  vision  Is: 

Ct  »  1.75  01/2  +  190/B2,  (0  >  0.8  deg) 

where  3  is  the  angle  subtended  by  the  target 
(arc  min.),  0  Is  the  off-axis  angle,  and  Cb  is 
percent  contrast  at  threshold.  It  is  clear  from 
these  formulae  that  the  effects  of  target  con¬ 
trast  depend  to  a  great  extent  on  the  position 
within  the  pilot's  visual  field.  However, 
atmospheric  visibility  will  markedly  affect  the 
level  of  target  contrast  that  Is  actually  per¬ 
ceived  by  the  pilot.  The  level  of  target  con¬ 
trast  that  actually  reaches  the  pilot's  eyes  Is 
the  effective,  as  opposed  to  the  Inherent, 
contrast  of  the  target.  Effective  contrast  in 
BRAWLER  Is  modeled  as  an  exponential  function  of 
Inherent  target  contrast  as  follows: 

c  ,  c<3  e-3-912  R/V 

where:  c  =  effective  target  contrast, 

c0  =  Inherent  target  contrast, 

R  *  range  of  target, 

V  =  atmospheric  visibility. 

Inherent  target  contrast  values  of  0.5  and  9.0 


34 


were  used  In  this  study.  Atmospheric  visibility 
was  set  at  20  nm  for  all  engagements.  Given 
that  the  effective  contrast  of  the  target  Is 
sufficient  for  detection  to  occur,  BRAWLER 
estimates  the  probability  of  detection  In  a 
single  glimpse  to  be  a  normal  ogive  function  of 
the  ratio  of  effective  to  threshold  contrast  for 
targets  subtending  any  given  visual  angle. i5' 

Simulation  of  resolution  effects  In  BRAWLER 
Is  based  on  the  apparent  size  of  the  target 
derived  from  the  angle  subtended  by  the  target 
at  a  given  range.  If  the  total  surface  area  of 
the  target  Is  not  sufficient  to  subtend  the 
minimum  visual  angle  (resolution  values)  used, 
the  target  would  remain  undetected.  Note  that 
the  value  8  (angle  subtended)  In  the  formula  for 
contrast  threshold  will  have  an  Inverse  effect 
on  threshold  detection  probability.  The  higher 
the  visual  angle  subtended  by  the  target,  the 
lower  the  contrast  threshold  required  for  target 
detection. 

Visual  search  characteristics  will  also 
affect  the  likelihood  that  a  target  within 
visual  range  will  be  detected.  BRAWLER  models 
the  pilot's  visual  search  pattern  by  dividing 
the  total  visual  field  (excluding  areas  masked 
by  the  cockpit)  Into  eight  sectors.  Sectors  ara 
searched  essentially  at  random  with  the  con¬ 
straint  that  the  same  sector  Is  not  searched 
again  until  at  least  one  other  sector  is 
sampled.  The  simulated  pilot  searches  each 
sector  for  2.5  sec.  with  no  more  than  150  msec, 
per  fixation.  Once  the  target  Is  detected,  a 
visual  tracking  algorithm  based  upon  the  optimum 
control  modeli®'  is  Initiated  while  the  target 
Is  In  view. 

Procedure 

The  purpose  of  using  BRAWLER  was  to  deter- 
minf  the  kinds  of  effects  display  resolution 
might  hove  on  engagements  In  a  manned  flight 
simulator  and,  therefore,  the  kinds  of  skills 
that  could  he  trained.  Since  display  resolution 
In  effect  limits  what  the  pilots  will  see  In  an 
engagement,  the  visual  acuity  of  the  "pilots"  In 
BRAWLER  was  altered  to  simulate  resolution 
acuities  of  2  and  4  arc  min.  These  values  were 
chosen  to  permit  comparison  of  resolution 
effects  for  display  systems  of  the  future  where 
2  arc  min.  may  be  considered  the  ultimate  goal 
of  a  system  designer.  Target  inherent  contrast 
values  of  0.5  and  4.0  were  also  simulated  in 
this  study.  The  contrast  value  of  0.5  approxi¬ 
mates  the  expected  contrast  of  targets  in  the 
real-life  engagements.  A  high  contrast  of  9.0 
was  chosen  to  Investigate  potential  trade-offs 
between  resolution  and  contrast  In  display 
design.  All  engagements  were  run  as  within 
visual  range  (WVR)  scenarios,  i.e.,  radar 
Intercept  was  not  possible. 

Engagement  Scenarios 

For  the  sake  of  simplicity,  only  BRAWLER 
data  on  a  one  vs  one  engagement  will  be  pre¬ 
sented  here.  The  aircraft  simulated  were  Air 
Force  F-15  fighters  equipped  with  radar  and 
Infrared  guided  missiles  and  guns.  At  the  start 
of  each  engagement  the  aircraft  were  situated 
with  reciprocal  headings  (head-on  approach)  and 


offset  by  1  nm,  with  a  slant  range  of  8  nm. 
Visual  acuity  values  for  a  given  engagement 
scenario  were  either  2  or  4  arc  min.  for  both 
pilots.  All  scenarios  assumed  dayligRt, 

unlimited  visibility  conditions.  As  with  real 
life  engagements,  some  variability  In  perfor¬ 
mance  is  expected  from, , on’  engagement  to  the 
next.  To  assure  reliability  of  tne  results,  25 
engagements  were  run  for  each  factorial  combina¬ 
tion  of  resolution  and  contrast  under  each  of 
the  two  (Head-On  or  Right-Angle)  Intercept 
conditions. 

RESULTS 

Results  of  the  BRAWLER  engagement  runs  were 
analyzed  for  effects  of  differences  In  simulated 
system  resolution  and  target  inherent  contrast 
on  average  detection  range,  opportunities  for 
early  shots,  advantage  of  first  slghter,  and  the 
type  of  weapons  used.  These  measurements  were 
chosen  because  they  would  reflect  the  general 
nature  of  the  engagements  and  are  reasonably 
accurate  Indicators  of  the  types  of  skills  that 
would  be  excerclsed.  The  results  were  analyzed 
separately  for  the  Head-On  and  Right-Angle 
engagement  scenarios. 

Detection  Range 

The  average  range  at  which  an  opponent  was 
detected  under  the  four  conditions  of  system 
resolution  and  target  contrast  are  shown  In 
Figure  2  for  the  two  engagement  scenarios.  The 
average  Head-On  detection  range  for  the  2  arc 
min.  case  Is  14,800  ft.  (S.D.  »  3,200  ft.)  and 
for  the  4  arc  min.  case  It  Is  11,100  ft.  (S.D.  » 
1,300  ft.)  with  a  target  contrast  at  0.5.  When 
the  target  contrast  Is  very  high  (9.0),  the 
average  detection  increases  to  20,300  ft.  (S.D. 
»  1,200  ft.)  for  the  2  arc  min.  resolution  case 
but  has  little  effect  on  the  4  arc  min.  case.  A 
similar  pattern  of  results  occurs  in  the  Right- 
Angle  engagement  scenarios.  The  Increase  In 
target  contrast  did  not,  as  might  be  expected. 
Improve  Initial  target  detection  in  the  poor 
resolution  runs.  A  doubling  of  resolution  from 
2  to  4  arc  min.  Improves  detection  but  only  by 
an  average  of  3,700  ft.  in  the  low  contrast 
conditions.  The  effects  of  resolution  and  con¬ 
trast  on  detection  range  are  reflected  In  the 
opportunity  for  early  shots. 

Early  Shots 

The  proportion  of  engagements  In  which 
weapons  were  fired  in  the  first  phase  of  the 
engagement  Is  shown  in  Figure  3.  The  improve¬ 
ment  In  detection  range  within  increased  acuity 
accounts  for  an  increase  of  onV  124  In  shots 
fired  during  the  Initial  phase  of  the  Head-On 
engagement  with  low  contrast.  With  Increased 
target  contrast  from  0.5  to  9.0,  early  shots 
rise  dramatically  from  16<  to  80X  of  the  engage¬ 
ments  in  the  2  arc  min.  case.  As  expected  from 
the  data  on  detection  range.  Improved  contrast 
had  no  significant  effect  on  early  shot  opportu¬ 
nities  In  the  Head-On  engagements.  The  pattern 
of  results  for  early  shots  as  with  detection 
range  data  Is  similar  for  both  scenario  types 
run. 


The  relatively  small  effects  on  detection 
range  and  early  shot  opportunities  resulting 
from  a  doubling  of  target  resolution  suggest 
that  only  a  small  gain  In  training  utility  Is 
achieved  for  this  critical  phase  of  air  combat 
engagements.  Attaining  positional  and  energy 
advantage  over  an  opponent  aircraft  early  In  an 
engagement  are  often  the  most  Important  factors 
In  determining  the  outcome  of  an  engagement. 
The  Increases  In  the  range  of  target  detection 
In  this  study  which  resulted  from  Improved 
target  resolution  were  not  of  sufficient  magni¬ 
tude  to  markedly  affect  the  positional  advantage 
of  either  combatant.  As  a  result,  the  opportu¬ 
nities  to  fire  weapons  early  In  the  engagement 
are  roughly  equivalent  for  all  engagements  run. 

Advantage  of  First  Sigh ter 

In  general,  air  combat  engagement  outcomes 
favor  the  pilot  who  sights  his  opponent  first. 
This  first  slghter  advantage  Is  reflected  In  the 
frequency  of  engagement  kills  shown  In  Figure 
4.  Note  that  these  kills  occur  In  the  secondary 
phase  of  the  engagements,  l.e.,  after  the  air¬ 
craft  have  passed  each  other.  The  high  kill 
frequency  In  the  secondary  phase  is  due  to  the 
low  proportion  of  early  shot  opportunities  in 
the  initial  phase  of  combat. 

Poorer  target  resolution  had  the  effect  of 
Increasing  the  advantage  to  the  first  slghter  In 
the  Head-On  engagements.  For  the  low  target 
contrast  condition,  the  proportion  of  kills  by 
the  first  slghter  Increases  from  64X  for  2  arc 
min.  of  resolution  to  79X  for  4  arc  min.  of 
resolution.  In  the  high  contrast  condition,  the 
effect  of  poorer  resolution  is  even  greater. 
The  proportion  of  kills  increases  from  54*  for  2 
arc  min.  to  100X  for  4  arc  min.  Target  contrast 
effects  had  generally  little  effect  on  first 
slghter  advantage  for  2  arc  min.  case  but  sub¬ 
stantially  increased  the  advantage  for  4  arc 

min.  of  resolution. 

The  differential  effects  of  target  resolu¬ 
tion  and  contrast  for  the  two  types  of  engage¬ 
ment  scenarios  on  the  relative  advantage  to  the 

first  slghter  are  due  to  several  factors.  The 
nature  of  a  Head-On  intercept  Is  such  that,  at 
the  longer  detection  ranges  occurring  with  2  arc 
min.  the  difference  in  detection  time  for  both 
combatants  Is  sufficiently  small  to  permit 

defensive  maneuvering  by  the  second  aircraft, 
thus  lowering  the  advantage  to  the  first 

slghter.  With  the  decreased  detection  range  at 
the  lower  resolution  (4  arc  min.),  a  higher 

proportion  of  engagements  occur  In  which  only 

one  aircraft  sights  the  other.  The  higher 

contrast  Increases  the  frequency  of  this 
occurrence.  In  the  Right-Angle  scenario  overall 
detection  range  Is  much  less  with  an  Increased 
frequency  of  cases  In  which  only  one  aircraft 
sights  the  other.  This  Is  due  to  the  tail-chase 
maneuver  that  Is  typically  executed  by  the  first 
slghter  In  these  engagements.  Higher  resolution 
In  the  type  of  engagement  had  the  effect  of 
increasing  the  time  that  the  first  slghter  has 
to  achieve  a  positional  advantage  (tail-chase) 
over  his  opponent.  This  resulted  In  a  higher 
proportion  of  cases  In  which  the  first  slghter 
was  never  detected  In  the  engagements  run  with  2 
arc  min.  of  target  resolution.  Improved  target 


contrast  offset  the  advantage  to  the  first 
slghter  by  permitting  a  higher  probability  that 
the  first  slghter  would  be  detected  by  the 
second  aircraft.  The  Improved  detection  by  the 
second  aircraft  resulted  In  defensive 
maneuvering  and  a  lower  lethality  of  early  shots 
fired  by  the  first  slghter. 

Weapons  Selection 

These  differential  affects  of  target 
resolution  and  contrast  for  the  two  scenarios 
are  also  found  in  the  selection  of  weapons 
during  the  engagements.  In  Figure  5,  the 
proportion  of  missile  kills  for  target  contrast 
and  resolution  values  are  shown  for  the  two 
scenarios.  In  the  Head-On  engagements  the 
proportion  of  kills  due  to  missiles  Increases 
substantially  as  target  resolution  is  reduced 
from  2  arc  min.  to  4  arc  min.  The  difficulties 
in  achieving  positional  advantage  with  reduced 
target  resolution  results  in  a  higher  frequency 
of  selecting  a  missile  over  a  gun  solution  In 
these  engagements.  In  general,  the  shorter 
detection  associated  with  the  Right-Angle 
scenario  eliminated  any  influence  of  target 
resolution  on  weapons  selection.  That  Is  to 
say,  the  vast  majority  of  engagements  were 
missiles.  Increased  target  contrast  has  the 
same  effect  on  weapons  selection  for  both  types 
of  engage-  ments.  An  Increased  frequency  of 
missile  selection  occurred  consistently  with 
Increased  target  contrast.  Engagements  run  with 
high  target  contrast  resulted  in  much  shorter 
differences  In  time  required  for  the  two 
aircraft  to  detect  one  another.  This  reduced 
time  differential  generally  favors  a  missile 
shot. 

DISCUSSION 

The  purpose  of  this  study  was  to  Investigate 
variations  In  target  resolution  and  contrast  in 

simulated  dir  combat  engagements  with  the  Intent 
of  deriving  Information  which  would  assist  in 
the  design  of  visual  displays  for  manned  flight 
simulators  in  air  combat  training  programs.  The 
results  of  the  study  Indicate  that  Increased 
target  resolution  had  only  a  small  effect  on 
Initial  target  detection.  Despite  the  fact  that 
these  resolution  values  are  representative  of 
design  goals  In  simulators,  the  differences  In 
the  critical  early  phase  of  an  air  combat 

engagement  that  result  from  improved  target 
resolution  are  small.  The  training  value  for 
display  configurations  with  a  resolution  beyond 
4  arc  min.  Is  doubtful  but  Increased  resolution 
to  2  arc  min.  does  not  result  In  substantial 

gains  in  the  early  phase  of  engagements  when 

measured  by  average  detection  range  or  early 
shot  opportunities.  A  substantially  greater 
benefit  results  from  improved  target  contrast  in 
the  initial  phase  of  air  combat.  This  Is 
largely  due  to  the  fact  that  Initial  detection 
of  an  opponent  aircraft  will  more  likely  occur 
In  the  pilot's  visual  periphery  where  acuity  Is 
poorer  but  responsiveness  to  contrast  Is  much 
greater. 

In  subsequent  phases  of  air  combat  engage¬ 
ments,  resolution  plays  a  more  Important  role. 
Poorer  resolution  clearly  results  In  some 
scenarios  In  which  the  first  slghter  will  have 


an  overwhelming  advantage.  The  advantage  Is  so 
large  as  to  allow  little  opportunity  for  defen¬ 
sive  maneuvering.  Poorer  resolution  also 
resulted  In  an  unreasonably  high  frequency  of 
missile  engagements.  In  some  cases  eliminating 
the  gun  as  a  viable  option.  In  training  for 
positional  advantage  and  weapons  selection,  a 
lower  resolution  display  would  seem  to  be  a  poor 
choice.  However,  in  some  engagements  such  as 
the  Right-Angle  Intercept  resolution,  differ¬ 
ences  have  little  impact. 

Target  contrast  enhancement  can  be  a  useful 
device  to  Improve  Initial  detection  and  thereby 
permit  some  training  with  poorer  resolution 
displays.  Despite  the  substantially  different 
contrast  values  used  In  this  study,  the  effects 
of  enhanced  target  contrast  on  the  qualitative 
nature  of  the  engagements  after  Initial  detec¬ 
tion  is  not  great.  In  general,  the  contribution 
of  target  contrast  In  air  combat  training  will 
depend  upon  the  extent  to  which  location  of 
targets  In  visual  periphery  (as  In  Initial 
detection)  Is  deemed  critical. 

This  study  has  demonstrated  the  advantage  of 
examining  the  potential  Impact  of  simulator 
display  characteristics  on  air  combat  training 
by  using  computer  modeling  techniques.  The 
changes  In  the  nature  of  air  combat  engagements 
with  variations  on  target  resolution  and  context 
Is  useful  In  determining  the  type  of  skills  that 
can  be  trained  In  simulators  having  these  dis¬ 
play  features.  The  use  of  computer  . modeling 
techniques  In  studying  simulation  fidelity 
requirements  has  the  potential  to  provide 
procurement  agencies  with  objective  data  on  the 
benefits  of  training  device'  as  a  part  of 
cost-benefit  analysis  prior  to  purchase. 
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CENTRAL  STATUS  ARRAYS 
(PHYSICALLY  OBSERVABLE  STATE  VARIABLES) 
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Figure  1.  Conceptual  Representation  of  Information  Flow 
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Figure  2.  Average  detection  range  for  Head-on  and  Right  Angle  engagement 
scenarios  as  a  function  of  pilot  acuity  and  target  inherent  contrast. 
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Figure  3.  Percent  engagements  with  weapons  fired  during  first  phase  (early  shots) 
as  a  function  of  pilot  acuity  and  target  inherent  contrast. 
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jrcent  of  kills  In  engagements  due  to  missiles  as  a  (unction  of  pilot  acuity 
and  target  inherent  contrast. 
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ABSTRACT 

The  training  of  gunners  for  man-portable  tactical  missile  systems  (such  as  Dragon) 
currently  requires  the  gunner  to  track  a  vehicle-mounted  target  and  field  !R  source  beacon. 
This  means  that  large  areas  are  required,  approximating  tactical  target  ranges.  These 
space/faclllty  requirements  limit  training  opportunities.  To  Increase  opportunities  while 
eliminating  (or  reducing)  space  requirements,  a  simulation  device  1$  required  that  displays 
battlefield  scenario  Images  In  the  gunner's  sight.  Currently,  simulator  technology  can 
optically  generate  simulated  battlefield  scenes  via  a  computer  controlled  display  screen. 
However,  the  high  cost  and  complexity  of  such  a  system  Is  prohibitive  when  used  In  a  tactical 
training  scenario.  This  paper  describes  a  low-cost,  microprocessor-based  training  device  to 
overcome  these  drawbacks.  The  major  component  of  the  system  Is  a  high  resolution  graphic 
display  sub-system  that  generates  Images  via  multiple  gray-level  video  signals  displayed  on  a 
Cathode  Ray  Tube  (CRT).  The  Image  data  Is  stored  In  memory  and  accessed  via  software.  Gunner 
tracking  movements  are  measured  and  used  to  modify  the  position  of  the  displayed  Image,  thus 
realistically  simulating  tactical  scenes.  The  gunner's  performance  Is  evaluated  by 
determining  tracking  errors  relative  to  target  llne-of-slght  (LOS)  and  comparing  these  errors 
to  established  error  limits.  Since  the  system  will  be  small  and  relatively  Inexpensive,  It 
will  readily  lend  Itself  to  classroom  or  field  training. 


INTRODUCTION 

The  U.S.  Army's  Dragon  missile  system  Is  a 
man-portable  anti-tank  weapon  that  gives  the 
Infantryman  the  capability  t*  destroy  armored 
vehicles.  The  air  vehicle  Is  launched  frojp  a 
recoil  ess  launch  tube  supported  by  a  bipod  and 
the  gunners  right  shoulder.  The  gunner  observes 
and  tracks  the  target  through  either  visible 
optics  or  a  thermal  Imaging  device  coupled  with 
the  missile  tracking  system.  Target  tracking  Is 
maintained  from  just  prior  to  launch  until  the 
air  vehicle  Impacts  the  target.  Target  ranges 
extend  to  1000  meters.  Figure  1  Illustrates 
typical  Dragon  usage. 

Presently,  training  for  Dragon  gunners  Is 
accomplished  using  either  a  day  or  night  tracker 
along  with  a  Launch  Effects  Trainer  (LET)  and  a 
performance  measuring  device,  the  Monitoring 
Set.  This  training  requires  the  use  of  a 
physical  range  out  to  1000m  and  a  vehicle  mounted 


FIGURE  1:  TYPICAL  DRAGON  U8AGE 


Infrared  beacon  so  that  the  gunner's  tracking 
errors  can  be  measured.  These  errors  are 
compared  against  error  limit  curves  which  are  a 
function  of  time.  An  error  that  exceeds  the 
limits  for  a  predetermined  length  of  time  Is 
scored  as  a  target  miss.  A  major  drawback  of 
this  training  Is  the  requirement  for  target 
ranges  to  1000m  and  the  associated  costs  of  range 
operation  and  maintenance.  Thus,  training  Is 
limited  to  Army  bases  where  this  type  range 
facility  exists. 

Effective  Initial  training  and  periodic 
requallflcatlon  require  realistic  scenarios  In 
the  gunner  sight.  This  scenario  requirement  Is 
especially  critical  In  night  tracker  training 
because  of  the  widely  variable  target  appearance. 
Targets  vary  In  size,  shape,  density  and  contrast 
In  the  thermal  night-sight  display.  When  used  In 
the  daytime,  the  thermal  Images  take  on  yet 
another  set  of  variables.  Therefore,  the  full 
spectrum  of  target  recognition,  range  estimation, 
and  target  tracking  exercises  are  very  difficult 
to  Implement  repeatedly  In  a  training  program  and 
become  a  second  major  drawback  to  present  train¬ 
ing  methods. 

These  drawbacks  Indicate  the  need  for  a 
simulator  to  generate  sight  scenarios  In  a 
controlled,  repeatable  and  realistic  fashion. 
Realism  dictates  an  Interactive  display  that 
senses  gunner  movements  and  modifies  scene 
positions  accordingly.  The  scene  simulations 
must  be  easily  Interchangeable,  thus  allowing 
varied  scenarios  Including  both  stationary  and 
moving  targets.  The  system  must  evaluate  gunner 
aiming  errors  and  score  the  gunners  tracking 
performance.  The  score  must  be  displayed  and/or 
printed.  The  system  should  also  be  self  con¬ 
tained  and  portable  for  field  use. 
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Future  man-portable  tactical  missile  systems 
will  utilize  day-night  Imagers  as  the  gunner's 
sighting  device  because  of  their  Inherent 
performance  advantage  under  adverse  conditions. 
These  systems  will  need  effective  training 
devices,  such  as  the  one  described  here,  which 
can  easily  be  updated  to  meet  their  requirements. 


DESIGN  CONCEPT 

System  Overview 

The  Automated  Gunner  Performance  Evaluator 
system  block  diagram  Is  shown  In  Figure  2.  The 
system  Is  based  on  off-the-shelf  microprocessor 
hardware.  The  fl-BIt  Central  Processing  Unit 
(CPU),  operating  system  Read  Only  Memory  (ROM) 
and  scratch  pad  Random  Access  Memory  (RAM)  form 
the  nucleus  of  the  hardware.  The  operating 
system  ROM  stores  the  firmware  to  control  system 
functions,  such  as  Image  display  control  and 
gunner  position  updates  during  a  simulated 
flight,  and  gunner  evaluation  and  scoring  after 
the  fight.  The  Image  data  ROM  contains  the  scene 
data  for  a  particular  simulation  and  Is 
Interchangeable.  The  display  subsystem  stores 
and  decodes  the  currently  displayed  Image.  The 
gunner  Interface  Inputs  tracking  position  data  to 
the  system.  The  scoring  display  and/or  printer 
Inform  the  gunner  and  Instructor  of  the  gunner's 
performance. 

Scene  Simulation 

The  simulation  of  tactical  scenarios  in  the 
gunner  sight  Is  Implemented  using  digital  data 
that  Is  compacted  and  stored  In  ROM.  A  portion 


of  this  data  Is  transferred  to  the  display 
subsystem  where  it  Is  decoded  Into  multiple  gray 
levels  per  picture  element  (pixel)  and  displayed 
on  a  CRT  using  rasterscan  graphics  techniques. 
The  CRT  replaces  the  normal  thermal  Imager 
electronics/display  and  Is  directly  viewed  by  the 
gunner.  The  storfed  Image  graphics  (SIG) 

technique  was  choosen  over-  a  calculated  computer 
generated  Image  (CGI)  for  several  reasons. 
First,  the  fact  that  the  gunner  operates  from  a 
fixed  position  for  any  one  flight  allows  stored 
Image  usage.  The  gunner's  range  to  the  target 
and  relative  roll  position  are  f1..ed  and  the  LOS 
only  changes  In  vertical  and  horizontal  angle. 
Thus,  a  stored  Image  that  Is  larger  In  horizontal 
and  vertical  dimensions  than  the  displayed  Image 
results  In  a  realistic  scene  scan  when  the 
gunner's  LOS  1$  sensed  and  used  to  position  the 
displayed  Image  within  the  stored  Image.  The 
stored  Image  dimensions  were  chosen  to  be  four 
times  the  displayed  Image  in  width  and  two  times 
the  displayed  Image  height  for  a  scene  area  eight 
times  the  displayed  Image.  These  dimensions  were 
chosen  because  the  primary  target  movements  are 
In  the  horizontal  plane  and  vertical  LOS 
movements  are  only  necessary  during  target 
acquisition. 

The  second  reason  for  choosing  the  SIG 
technique  Is  reduced  software  overhead.  The 
software  task  reduces  to  memory  data  transfers 
using  horizontal  and  vertical  offset  pointers. 
Calculated  CGI  techniques  use  complex 
mathematical  operations  and  require  very  high 
processing  speeds  to  operate  In  real  time.  The 
SIG  technique  allows  system  Implementation  using 
standard  microprocessor  hardware  that  Is 
Inexpensive  and  easy  to  develop. 


Fiouna  2  AUTOMATED  GUNNER  PERFORMANCE  EVALUATOR  HOCK  DIAGRAM 
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The  displayed  Image  Is  a  raster  scan  CRT 
Image  consisting  of  128  lines  and  128  pixels  per 
scan  line.  The  displayed  Image  requires  only 
4096  bytes  of  memory  due  to  the  data  compaction 
used.  Thus,  the  overall  scene  Is  stored  In  32768 
bytes  of  memory  (8X  Image  size).  The  portion  of 
the  scene  displayed  depends  on  the  value  of  two 
memory  pointers,  horizontal  and  vertical  offset, 
as  shown  In  Figure  3.  The  pointer  values  are 
controlled  by  the  horizontal  and  vertical  gunner 
position  sensors  described  below.  A  typical 
simulation  run  starts  with  these  pointers  set  to 
nominal  values  and  gunner  position  nulled  to 
zero.  Thus,  the  displayed  Image  Is  centred  In 
the  overall  scene.  When  the  gunner  has  assumed 
his  firing  position,  the  Instructor  starts  the 
simulation.  The  memory  pointers  are  now  modified 
by  the  gunner  position  data  as  he  moves  around  to 
acquire  and  track  a  target.  Realistic  Image 
movement  Is  then  simulated  within  the  limits  of 
the  overall  scene. 


TOTAL  SCENE 
IMAGE 


FIGURE  3.  SCENE  DATA  MAP  SHOWING  DISPLAYED  IMAGE 
DATA  OVERLAY 


Each  pixel  Is  represented  by  two  bits  of 
an  8-blt  byte  of  memory  data.  These  two  bits  are 
decoded  to  produce  four  CRT  Intensity  levels. 
These  levels  are  black,  white  and  two  Inter¬ 
mediate  gray  levels.  Intensity  approximation  to 
four  levels  was  chosen  as  adequate  because  of  the 
limited  Intensity  ranges  actually  displayed  In 
thermal  night  sights.  The  gray  level  decoding  Is 
hardware  Implemented  In  the  display  system  video 
data  decoder  (see  Figure  2). 

Two  bit  Intensity  data  per  pixel  results  In 
compacting  four  pixels  of  data  per  byte  of  memory 
data.  There  are  two  advantages  to  this  data 
formatting.  First,  only  32  bytes  of  memory  are 
accessed  per  raster  scan  line,  resulting  In  a 
time  of  2  microseconds  between  display  memory 
read  cycles.  Thus,  slower  memory  can  be 
utilized.  Secondly,  the  software  for 

transferring  data  to  the  display  memory  generates 
only  4096  memory  address  (128*  pixel  s/4  »  4096) 
bytes  which  reduces  software  overhead. 

The  only  disadvantage  of  this  pixel  data 
format  Is  that  the  displayed  Image  moves  In  byte 
or  four  pixel  steps  when  horizontal  Image 

movement  Is  required  If  the  software  uses  byte 
addressing  only  to  access  the  data.  This 

limitation  Is  overcome  using  hardware  so  that  bit 
manipulation  is  not  required  In  software.  The 

method  used  converts  the  data  format  from 


parallel  to  serial  and  selects  the  required  pixel 
offset  for  the  data  to  be  displayed.  The 
selection  Is  controlled  by  the  gunner  position 
error  data.  Thus,  Image  movement  can  be  resolved 
In  ohe  pixel  Increments  In  the  horizontal 
direction  by  special  hardware  and  byte  increments 
by  changes  In  data  addressing. 

The  scenes  simulated  must  be  Interchangeable 
to  allow  maximum  flexibility  of  the  simulator. 
Various  mass  storage  mediums  were  considered. 
The  need  for  a  portable  system  that  could  be  used 
In  the  field  under  diverse  environmental  condi¬ 
tions  Indicated  the  best  storage  medium  to  be  RON 
devices.  ROM's  are  readily  available  In  an  8192 
X  8  bit  word  size  and  only  four  memory  devices 
are  necessary  per  scene.  These  devices  were  also 
choosen  because  equivalent  erasable,  programnable 
ROM's  (tPROM's)  are  available  for  use  during 
development  work.  The  final  scene  modules  will 
be  packaged  In  an  easily  replaceable  plug-in 
cartridge  for  ease  of  use. 

Gunner  Interface 

The  Dragon  missile  system  uses  a  bipod  to 
support  the  front  of  the  launch  tube  (see  Figure 
1).  This  bipod  allows  movement  In  the  vertical 
and  horizontal  directions.  Gunner  movements  can 
be  sensed  by  attaching  sensors  to  measure  the 
relative  movements  at  the  swivel  Joint  of  the 
bipod  for  the  horizontal  axis  and  the  changes  In 
launch  tube  Inclination  for  the  vertical  axis. 
The  sensor  resolution  required  Is  approximately 
0.5  mllllradlan  (mr).  This  resolution  Is  most 
easily  obtained  using  an  Incremental  optical 
shaft  encoder  In  the  horizontal  axis  and  a 
pendulum  type  Inclinometer  In  the  vertical  axis. 
The  shaft  encoder  Is  used  with  a  preset  up-down 
counter  to  derive  position  data.  The  counter 
preset  Is  the  nominal  value  for  the  horizontal 
offset  pointer  and  Is  Initialized  to  this  value 
prior  to  a  simulation  run.  The  Inclinometer  Is 
Interfaced  to  the  microprocessor  via  an  A/D 
converter  and  the  Initial  value  at  the  start  of  a 
simulation  Is  substracted  from  all  subsequent 
data.  This  configuration  allows  easy  Initial 
setup  and  does  not  require  absolute  bipod 
positioning. 

The  microprocessor  CPU  accesses  the  gunner 
position  through  two  8-blt  Input  ports  fed  from 
the  up-down  counter  output  and  the  A/D  output. 
The  two  least  slgilflcant.  data  bits  In  the 
horizontal  axis  are  fed  to  the  video  decoder 
hardware  directly  to  select  the  pixel  offset  In 
the  data  word,  thus  Implementing  Individual  pixel 
resolution  of  display  movement.  The  position 
data  Is  read  at  1/30  sec.  Intervals,  synchronized 
with  the  vertical  sync  pulse  of  the  video 
generator.  The  data  acquired  Is  used  to  set  the 
vertical  and  horizontal  offset  software  pointers 
and  update  the  displayed  Image  position  within 
the  overall  scene. 

Gunner  Performance  Evaluation 

The  value  of  the  vertical  and  horizontal 
offset  pointers  versus  time  during  a  run  are  a 
direct  Indication  of  the  gunners  aim  point  since 
they  have  a  fixed  relation  to  the  center  of  the 
display  area.  Therefore,  a  data  table  of  these 
pointer  values  can  be  used  to  evaluate  gunner 
performance. 


Man-portable  tactical  missile  systems  such 
as  Dragon  that  require  target  tracking  until 
Impact  have  aiming  error  limits  that  vary  with 
time  after  launch.  The  general  shape  of  these 
curves  Is  Illustrated  In  Figure  4.  The  initially 
wide  limits  allow  for  launch  transients.  If, 
prior  to  the  target  Impact,  the  gunner's  error 
exceeds  these  limits  longer  than  a  predetermined 
amount  of  time,  the  gunner  will  miss  the  target. 
Also,  the  error  must  not  exceed  target  dimensions 
at  Impact. 


LAUNCH 

FIGURE  4:  TYPICAL  GUNNER  AIMING  ERROR  LIMIT  CURVES 

'  At  the  end  of  a  run,  the  gunner  aiming  error 
Is  computed  by  subtracting  the  target  position 
from  the  gunner  aim  point  data.  The  aiming  error 
Is  then  compared  against  the  limit  curve  data  to 
score  the  gunner. 

The  gunner's  score  will  be  displayed  on  the 
front  panel  of  the  evaluation  set  as  a  HIT  or 
MISS.  If  a  MISS  Is  scored,  the  time  that  the 
error  exceeded  the  limits  Is  also  displayed.  If 
a  HIT  Is  scored,  the  aim  point  data  Is  displayed 
to  Indicate  actual  Impact  point. 

Provisions  are  made  for  an  optional  scoring 
printer.  In  addition  to  printing  the  scoring 
data  noted  above,  the  printer  will  provide  a 
graph  of  the  vertical  and  horizontal  aiming  error 
versus  time  for  further  gunner  evaluation. 
Corrective  action  mesages  will  also  be  printed  in 
response  to  various  types  of  errors. 


DEVELOPMENT  SYSTEM  HARDWARE 
Microcomputer  System 

The  microcomputer  used  Is  an  Ohio  Scientific 
(OSI)  model  C3-0EM.  This  system  utilizes  OSI's 
triple  processor  board  that  allows  easy  program 
development  on  6502,  Z80  and  6800  type 
processors.  The  RAM  Is  configured  as  48K  of 
static  user  memory.  Program  entry  and  display  Is 
through  a  serial  terminal.  Two  8"  floppy  disk 
drives  are  connected  to  give  about  500k  bytes  of 
program  and  data  storage.  A  printer  Is  connected 
to  the  system  for  hard  copy  printouts  when 
necessary. 

The  video  subsystem  Is  a  custom  designed 
circuit  that  Incorporates  the  pixel  Intensity 
decoding  and  pixel  selection  hardware  referred  to 
above.  The  display  memory,  memory  multiplexer, 
display  memory  address  scanner  and  special  video 
driver  are  also  on  this  board.  Figure  5 
Illustrates  this  hardware  setup. 


Video  Scene  Data  Acquisition 

The  Initial  scene  data  being  used  In 
developmental  work  Is  acquired  via  a  television 
camera.  This  data  is  suitable  for  simulation  of 
day  tracker  scenes  and  hardware  checkout.  Data 
Is  acquired  using  a  Blomatlon  model  1010  data 
digitizer  and  a  custom  sysnchronlzlng  circuit  to 
digitize  the  video  on  a  line-by-line  basis  of  the 
frame  scan. 

Simulations  of  thermal  nlghtslght  scenes 
require  gathering  data  using  a  thermal  tracker 
video  source.  Data  Is  oathered  in  a  similar 
fashion  using  the  Blomatlon  1010  and  a  second 

custom  synchronizing  circuit. 

The  raw  digital  video  data  Is  stored  In 

memory  of  the  Blomatlon  1010.  An  Input/output 
(I/O)  port  of  the  development  system 
microcomputer  Is  used  as  an  Interface  to  obtain 
the  data  from  the  Blomatlon  1010  memory.  The 
digital  data  Is  processed  by  a  software  routine 
that  senses  and  Ignores  sync  pulses.  The 

threshold  levels  for  the  four  gray  levels  are 
controlled  In  the  same  routine.  Each  pixel  Is 

compared  to  the  thresholds  and  assigned  a  gray 
level.  The  software  also  accumulates  four  pixels 
into  one  data  byte  and  then  stores  the  data  In  a 
file  on  disk  memory  for  usage  later. 


PROGRESS  TO  DATE 

Video  Subsystem 

The  video  subsystem  described  above  has  been 
Implemented  and  Is  totally  functional.  A  standard 
television  monitor  Is  used  to  display  the  video. 
Figure  6  shows  a  test  pattern  generated  to  verify 
all  gray  levels  are  present,  and  generated  In  the 
proper  sequence. 

Scene  Acquisition  and  Data  Reduction 

The  television  camera  and  the  Blomatlon  1010 
have  been  successfully  used  to  digitize  station¬ 
ary  target  test  scenes.  The  threshold  and  com¬ 
pacting  software  routines  properly  reduce  the 
data,  generating  data  files  for  these  scenes. 
The  picture  resolution  and  Intensity  approxi¬ 
mations  are  compatible  with  the  performance  of 
the  existing  Dragon  thermal  night  sight 
(AN/TAS-5) . 

Gunner  Interfacing 

The  gunner  position  sensors  are  In  the  final 
design  phase.  The  up-down  counter  and  I/O  port 
have  been  functionally  verified  using  an 
oscillator,  proper  gating  and  "joy-stick"  to 
simulate  the  Incremental  shaft  encoder.  The  only 
remaining  tasks  are  physical  Integration  of  the 
shaft  encoder  to  the  bipod/  launch  tube  and  the 
electrical  connections.  This  effort  should  be 
compl eted  by  the  fourth  quarter  1 981 . 


AREAS  OF  FURTHER  INVESTIGATION 

Both  moving  and  stationary  targets  are 
needed  If  the  full  spectrum  of  tactical  targets 
are  to  be  simulated.  Investigations  to  implement 
moving  targets  will  consider  scene  data 
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substitution  toehnlquts  utilizing  «  separate 
target  memory  and  appropriate  software  pointers 
to  position  the  target. 

Simulation  of  battlefield  obscurants  such  as 
smoke  and  explosive  round  Impacts  would  enhance 
the  realism  of  the  simulation.  Another 
enhancement  would  be  simulation  of  an  explosion 
at  the  target'  If  the  gunner  gets  a  target  hit. 
Investigations  to  Implement  these  enhancements 
will  also  consider  scene  data  substitution 
techniques.  The  additional  software  overhead 
necessary  to  Implement  moving  targets  and  the 
additional  enhancaments  may  degrade  real-time 
performance  of  the  main  scene  simulation.  If  so, 
a  second  microprocessor  will  be  considered  that 
would  handle  all  dynamic  Images.  Tradeoff 
studies  will  balance  additional  hardware  com¬ 
plexity  against  desirability  of  the  enhancements. 

Four  gray  level  Intensity  approximations  can 
result  In  less  than  optimum  Images  under  some 
conditions.  Therefore,  pixel  averaging  will  be 
Investigated  to  determine  If  Image  Improvement 
results.  The  techniques  Investigated  will  most 
probably  use  hardware  In  the  video  decoder-driver 
so  that  software  overhead  Is  not  effected. 

McDonnell  Douglas  training  studies  have 
shown  that  the  gunner/instructor  communication 
and  training  effectiveness  Is  greatly  enhanced  If 
the  Instructor  can  actually  see  the  same  sight 
picture  as  the  gunner.  Therefore,  a  CRT  display 
for  the  Instructor  will  be  added.  The  software 
for  using  this  display  to  replay  the  simulation 
run  will  be  developed,  thus  allowing  the  gunner 
to  see  any  tracking  mistakes  he  has  made.  The 
possibility  of  displaying  scoring  Information  and 
aiming  error  plots  on  the  Instructor  CRT  also 
will  be  Investigated. 


CONCLUSIONS 

Training  equipment  to  simulate  thermal  night 
sight  Images,  such  as  the  Dragon  AN/TAS-8,  does 
not  presently  exist  In  the  Army  Inventory.  The 
Automated  Gunner  Performance  Evaluator  described 
In  this  paper  fills  a  need  for  a  cost  effective, 
small,  portable  night  sight  training  device  that 
can  be  deployed  for  field  usage  In  Initial  and 
skill  maintenance  training.  The  system  Is  Imple¬ 
mented  using  standard,  off-the-shelf  micropro¬ 
cessor  hardware,  thus  keeping  potential  produc¬ 
tion  costs  low*  The1  stored  Image  techniques  used 
to  generate  fpages  minimize  the  software  over¬ 
head,  allowing  standard  microprocessors  to  accom¬ 
plish  the  simulation.  The  Interchangeable  scene 
memory  feature  allows  generation  of  a  wide  spec¬ 
trum  of  Images  ualng  hardware  that  Is  reliable  In 
all  potential  field  environments. 
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This  paper  describes  the  full  color  out-the-windcw  visual  system  simualtor  for  the  Navy's  A7E 
Weapons  System  Trainer  (WST),  based  on  the  Vought  -  developed  Computer  Animated  Photographic 
Terrain  View  (CAPTV)  concept.  The  system  has  excellent  scene  realism  due  to  its  real-world 
photographic  data  base.  Any  geographic  area  can  be  'flown  over'  in  the  simulator  with  virtually 
no  restrictions  on  aircraft  attitude,  position,  altitude, heading  or  velocity.  The  flyspace  can 
be  increased  indefinitely  by  additions  to  the  terrain  library.  The  display  field  of  view  is 
moaularly  expandable  up  to  nearly  the  full  encircling  sphere  while  maintaining  resolution  of 
all  parts  of  the  scene. 


I.  INTRODUCTION 

The  need  to  provide  scene  realisr.  in  Visual 
Systems  for  flight  si-milators  is  well  recognized. 
To  this  end,  real  world  imagery  is  used  in  Vought 
developed  Computer  Animated  Photographic  Terrain 
View  (CAPTV)  Concept.  The  first  successful  CAPTV 
system  (1)  employed  monochrome  aerial  photographs 
in  a  large  random  access  video  data  base,  which, 
through  computer  processing,  provided  smooth  high 
detail  simulated  visual  motion  cues  to  the  pilot 
trainee.  This  paper  describes  the  follow-on 
effort  at  Vought  to  provide  a  full  color  out-the- 
wlndow  visual  system  fer  the  ;lavy '  3  A-7E  Weapon 
System  Trainer . 


11.  ca:-t  concept 

The  basic  CAPTV  concept  assumes  an  array  of 
still  photographs  taken  from  an  airplane  that 
covers  the  gaming  area  defined  by  pilot  training 
requirements.  Photographs  are  taken  at  regular 
intervals  along  straight  and/or  cross  tracks. 

These  photos  are  scanned,  formatted,  and  stored  in 
a  bulk  storage  device.  As  the  pilot  'flies' 
through  these  photos  using  controls  similar  to  the 
ones  on  the  cockpit  panels,  they  are  retrieved 
from  the  storage  medium  for  display.  At  any 
given  instant,  knowing  the  pilot's  eyepoint  in 
apace,  the  photo  in  the  database  nearest  his 
location  is  stretched,  skewed,  rotated  and  trans¬ 
lated  in  a  piece-wise  continuous  mathematical 
transformation  such  that  the  transformed  photo 
would  overlay  a  different  photo  taken  from  the 
pilot's  eyepoint  (Fig.  1).  This  process  is  a 
continuous  one  each  frame  time  under  computer 
control  and  allows  the  introduction  of  smooth 
translation  into  e  basically  still  picture  set  for 
any  direction  of  travel  through  that  set.  During 
the  period  in  which  motion  is  taking  place  using 


a  given  fixed  scene,  the  computer  is  fetching  a 
new  appropriate  view  to  be  used  as  an  overlay 
substitute.  The  new  view  is  selected  by  a 
sophisticated  prediction  scheme  that  determines 
when  the  present  photo  must  be  discarded  in  favor 
of  a  new  photo.  It  is  important  to  understand 
that  the  transformation  process  does  not  cause 
unrealistic  distortions  of  the  viewed  scene.  The 
key  to  CAPTV  lies  in  its  unique  capability  to 
make  one  fixed  photo  serve  in  a  dynamic 
translation  situation  long  enough  to  fetch 
another  view  and  incidentally  perform  many  other 
functions  as  well. 


III.  OFF-LINE  IMAGE  GENERATION 

The  off-line  image  generation  of  the  color 
Visual  System  is  used  to  develop  the  data  for  the 
playback  system.  The  image  generation  consists 
of  the  following  subsystems. 

(a)  Aerial  Camera 

To  provide  for  the  360°  of  azimuth  coverage 
and  100°  of  elevation  (with  high  resolution 
throughout),  a  special  camera  (Fig.  2) 
incorporating  seven  lenses  with  associated  mirrors 
has  been  fabricated,  A  typical  exposure  on  a  9" 
color  film  is  shown  in  Fig.  3  (actual  photographs 
used  in  the  simulation  are  in  color).  Six  lenses 
capture  the  oblique  views  and  the  central  lens 
covers  the  straight  down  vertical  view.  The 
gaming  area  is  partitioned  to  contain  a  large 
number  of  'eyepoints'  distributed  in  several 
altitudes  in  several  straight  and  cross  tracks. 

The  photography  density  varies  inversely  with 
altitude  to  provide  the  required  coverage  for 
smooth  transition  from  scene  to  scene.  Objects 
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which  move  m  the  terrain  is  photographed  are 
treated  aa  atationary  anomalies  or  are  removed 
during  the  record  proceaa. 

(b)  Record  Processor  System 

The  record  processor  system  (Fig.  **)  converts 
the  photographic  image  into  video  signals  for 
recording  on  Video  Tape  Recorders  (VTR).  A  flying 
spot  scanner  is  used  to  scan  the  9"  film  roll  to 
provide  a  pixel  resolution  of  approximately  **000 
pixels  in  both  horizontal  and  vertical  directions. 
Because  of  the  size  of  the  film  and  separation  of 
the  seven  views,  scanning  is  done  in  steps.  All 
of  the  seven  views  are  assembled  in  a  pseudo-film 
plane  using  a  resolution-preserving  mathematical 
transformation.  The  video  in  the  primary  colors 
of  red  (R),  green  (G)  and  blue  are  digitized,  and 
formatted  such  that  each  terrain  scene  is  made  up 
of  several  NTSC  frames  (typically  72).  Appropriate 
sync  signals  are  Inserted  to  make  the  signals 
suitable  for  recording  on  standard  1  inch  video 
tape  recorders  and  laser  disks.  To  obtain  faster 
access  during  playback,  pieces  of  a  complete  scene 
are  stored  in  three  different  videotapes.  During 
the  record  process,  opportunity  is  afforded  to 
manipulate  the  luminance  gain  and  color  balance 
of  each  view  in  every  scene  so  that  a)  the  edges 
of  the  view  exposed  by  all  the  seven  lenses  match 
up  b)  successive  scenes  have  their  color  and 
brightness  matched.  The  key  to  this  processing  is 
a  luminance  and  color  corrector  block  together 


with  the  large,  fast  semiconductor  memory  called 
the  scene  storage  system  (SSS)  which  can  store 
upto  296  Rational  Television  System  Coesaieelon 
(NTSC)  frames. 

(c)  Photographic  Film  Edit  Syatem 

Using  highly  accurate  photogaametric 
techniques  (2),  the  eyepoint  of  every  scene  is 
determined.  This  Information  aide  in  the  proper 
sequencing  of  the  scanned  scenes  on  the  tape 
recorders.  The  knowledge  of  accurate  eyepoints 
also  allots  matching  one  distorted  scene  to 
another  when  both  scenes  are  viewed  from  the  same 
intermediate  pilot  position. 

(d)  Video  Disk 

The  data  base  recorded  on  the  video  tape 
recorders  is  transferred  to  video  disks  suitable 
for  playing  on  the  Disco  Vis loo  720  industrial 
optical  disk  player.  Each  of  the  video  disks  can 
store  up  to  54,000  standard  NTSC  frames  or 
approximately  700  color  visual  scenes.  Both  frame 
identification  numbers  and  Society  of  Motion 
Picture  and  Television  Engineers  (SMPTE)  Codes  are 
used  for  the  proper  sequencing  of  the  frames. 

New  data  bases  can  be  created  • imply  by  going 
through  the  above  steps  of  aerial  photography, 
scanning,  recording,  disc  mastering  and  photo- 
gammetry  in  the  proper  sequence.  Therefore,  there 
is  virtually  no  limit  to  the  expandability  of  the 
gaming  area. 


Figure  4.  Record  Processor  System 
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XV.  ON-LINE  PLAYBACK  SYSTEM 

The  real-time  on-line  playback  eyatea  (Fig. 
5)  consists  of  the  following  subsystem . 


Analog  storage  using  television  techniques  is 
a  practical  and  economical  alternative  to  digital 
storage  and  is  especially  suited  to  pictorial  data. 
The  purpose  of  the  storage  system  consisting  of  the 
sixteen  video  disks  is  to  provide  large  segments 
of  the  desired  data  under  control  of  the  host 
computer  in  anticipation  of  the  real  time  needs  of 
the  simulation.  The  bulk  data  is  distributed  over 
the  sixteen  disks  to  optimize  the  retrieval  of  the 
desired  data.  Any  randomly  accessed  TV  frame  on  a 
single  play-back  unit  can  be  located  and  made 
available  in  2  to  8  secs.  Adjacent  or  sequential 
frames  require  much  less  time  than  this,  however. 
The  players  operate  essentially  independently  and 


are  rotat tonally  synchronised  to  the  system 
timing  references  for  color  subcarrier  and 
horizontal/vertical  raster  scans.  In  operation, 
all  these  units  do  not  supply  data  at  the  same  time. 
Data  is  transferred  from  only  three  players  at  a 
time  while  the  others  are  searched  for  the 
anticipated  subsequent  scenes. 


The  Video  Digitizer  System  (VDS)  (Fig.  6)  is 
the  functional  unit  which  processes  analog 
composite  video  from  the  Video  Storage  System  (VSS) 
ana  sends  digitized  video  to  the  scene  storage 
system  (SSS).  The  VDS  has  three  identical  channels 
since  at  any  given  time  three  video  disc  players 
can  be  simultaneously  accessed.  The  quasi-NTSC 
video  from  the  VSS  is  read,  tiste  base  corrected, 
clasped  and  digitized  in  the  VDS.  Composite  video 
to  component  video  conversion  and  spatial 
conpression  of  the  digitized  video  are  carried  out 
by  the  luminance  and  chrominance  processors  in  an 
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Figure  5.  Visual  On-Line  Playback  System 
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Figure  6  Video  Digitizer  System 


effort  to  take  advantage  of  the  redundancy  present 
in  the  X  and  Q  components  of  the  color  signal  and 
to  save  memory  storage  space  in  the  SSS.  Hie  VDS 
also  provides  the  interface  between  the  visual 
system  computer  and  the  rest  of  the  visual  system 
hardware . 

(c)  Scene  Storage  System 

The  SSS  comprises  of  two  identical  sections - 
upto  128  tracks  each.  Each  track  has  the  capacity 
to  store  digital  video  information  including  color, 
present  in  one  NTSC  TV  frame  (1/30  sec.). 

Video  information  can  be  written  to  one  section 
while  it  is  read  from  the  second  section.  When 
fresh  data  is  required,  the  input  side  becomes  the 
output  side  and  vice  versa.  Three  channels  of  data 
can  be  input  to  the  SSS  and  the  SSS  in  turn  can 
provide  multiplexed  Y,  I  &  Q  digital  data  to  the 
U  cell  processor  systems  simultaneously. 

(d)  Cell  Processor  System  (CPS) 

The  cell  processor  system  receives  the  digital 
luminance  and  chrominance  data  corresponding  to  a 
scene  photograph  from  the  SSS.  Under  software 
control  it  selects  the  portion  which  is  visible  to 
the  pilot  through  one  windcw,  performs  the 
geometric  transformation  on  this  data  bb  required 
by  motion  of  the  pilot's  eyepomt,  and  generates 
analog  red,  green  and  blue  video  signals  for 


display.  Data  is  received  from  the  SSS  by  four 
input  interpolators  which  operate  Independently 
and  simultaneously  (Pig.  7).  Each  of  the  inter¬ 
polators  channels  the  Y,  I  &  Q  into  separate  data 
streams,  performs  low-pass-filtering,  and  inter¬ 
polates  or  decimates  as  required  to  provide  data 
at  the  proper  rate  to  be  loaded  into  the  cell 
memory.  Filtering  and  interpolation  are  done  in 
both  longitudinal  and  transverse  directions,  and 
the  parameter  controlling  these  operations  is 
supplied  by  the  visual  computer. 

The  cell  memory  consists  of  two  sections. 

One  section  may  be  read  for  display  while  the 
other  is  being  loaded  by  the  interpolators.  An 
output  multiplexer  selects  the  data  from  the 
appropriate  section  and  provides  it  to  the  output 
processor. 

The  addresses  for  the  Cell  Memory  are  provided 
by  the  Address  Generator.  The  read  addresses  are 
obtained  from  the  display-raster  pixel  coordinates 
by  a  transformation  corresponding  to  a  piecewise 
continuous  rotation,  skew,  and  magnification  in 
order  to  create  a  display  of  the  scene  data  as  seer 
from  the  pilot's  eyepoint. 

The  output  processor  receives  data  through  the 
memory  multiplexer,  performs  the  conversion  from 
Y,  I,  Q  format  to  Red,  Green,  Blue  format, 
performs  low  pass  filtering,  simulates  haze,  and 
inserts  a  blue  sky  above  the  horizon.  It  also 
provides  for  replacement  of  a  number  of  pixels  in 
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Figure  7  Cell  Processor  System 


the  output  scene  with  pixels  supplied  by  the 
Visual  Computer  and  for  insertion  of  a  light  point 
surrounded  by  a  black  square  for  the  automatic 
position  error  sensing  system.  Finally  the  R,  G, 
B  digital  data  streams  are  converted  to  analog 
signals,  filtered  and  provided  to  the  display 
along  with  a  composite  sync  signal. 


V.  IMAGE  D IS FLAY  SYSTEM 

The  image  display  system  consists  of  a 
multiple  rear  screen  television  projection  system 
providing  essentially  the  total  forward  hemisphere 
field  of  view  (FOV)  available  from  the  A7-E 
cockpit.  This  1b  obtained  by  Joining  together 
six  flat  projection  screens  into  a  single  mosaic. 
The  total  FOV  and  the  FOV  of  each  channel  is 
depicted  in  Figure  8.  Each  separate  rear  pro* 
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jection  channel  consists  of  an  875  line  General 
Electric  Light  Valve  TV  projector,  two  or  more 
folding  mirrors  and  a  lenticular  rear  projection 
screen.  The  overall  system  design  is  depicted  in 
figure  9, 


VI.  CONCLUSION 

CAPTV  offers  some  major  advantages  over  the 
CGI  visual  system.  Some  of  these  advantages  are 
its  ability  to  provide  scene-realism,  detail, 
3-dlmensionallty,  and  texture  that  is  limited 
only  by  photography  and  virtually  unlimited  for 
expandability  of  data  base.  The  real-time  play 
back  system,  when  added  to  the  existing  WST 
equipment,  will  provide  the  pilot  with  a 
continuous  display  of  the  gaming  area  as  presented 
on  a  hemispherical  rear  screen  projection  system. 
The  scope  of  the  gaming  area  is  large  enough  and 
flexible  enough  for  a  aim  lator  pilot  to  fly  a 
mission  with  total  maneuvering  freedom. 
Registration  with  the  radar,  FLIP,  and  projected 
map  display  in  the  '.VST  will  be  maintained 
regardless  of  the  simulated  aircraft  maneuvers 
throughout  the  flight. 
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ABSTRACT 


In  this  study  the  use  of  augmented  feedback  was  investigated  as  a  means  of 
training  low  altitude  perceptual  motor  flying  skills  In  a  flight  simulator.  Sixteen 
T-38  students  pilots  enrolled  In  Air  Force  undergraduate  pilot  training  participated 
as  subjects.  Eight  subjects  In  an  experimental  group  were  trained  to  fly  low  level 
In  a  simulated  A-10  aircraft  using  special  altitude  prompts  (lights  on  the 
glareshleld  and  auditory  tones  In  the  headset)  to  assist  them  In  discriminating 
altitude  cues  provided  In  the  simulated  visual  environment.  Eight  subjects  In  a 
control  group  received  training  Identical  to  that  of  the  experimental  group,  less 
prompting.  A  computerized  data  record  system  captured  a  continuous  record  of 
altitude,  vertical  velocity,  number  of  crashes,  and  other  performance  parameters  on 
each  of  eight  training  trials  and  two  test  runs  In  which  prompts  were  omitted.  All 
subjects  flew  a  total  of  ten  runs.  The  prompted  group  achieved  significantly  lower 
altitude  performance  on  two  of  four  critical  task  segments  compared  to  the  control 
group  during  the  training  trials.  However,  subjects  In  the  prompted  group  crashed 
significantly  more  times  per  trial  than  did  subjects  in  the  control  group  dur4ng  the 
training.  During  the  test  runs  performance  of  the  two  groups  for  altitude,  vertical 
velocity,  and  frequency  of  crashes  was  not  significantly  different.  The  results  of 
the  study  do  not  appear  to  warrant  continued  Investigation  of  this  technique  for  low 
level  training. 


PROBLEM:  TERRAIN  FLIGHT  SIMULATOR  TRAINING 

Lack  of  adequate  visual  scene  detail 
limits  the  usefulness  of  currently  available 
computer  generated  Imagery  (CGI)  for  training  low 
level  flight.  P-esent  levels  of  detail  and 
picture  resolution  are  Inadequate  to  produce 
desirable  representation  of  ground  patterns  and 
features.  Both  ground  textural  patterns  and 
vertical  objects  appear  to  be  used  as  primary 
visual  cues  by  pilots  In  judging  aircraft  height 
above  the  ground.  Since  present  CGI  limitations 
preclude  adequate  terrain  detail,  questions 
remain  as  to  how  to  manipulate  scene  content  and 
training  techniques  In  order  to  optimize  existing 
CGI  capabilities.  It  Is  to  be  hoped  that  such 
developments  can  compensate  to  some  extent  for 
the  current  lack  of  scene  fidelity. 

LOW  LEVEL  ENVIRONMENT  DEVELOPMENT: 

RELATED  RESEARCH 


Touchdown  Study 


Studies  i.  AFHRL/OT  have  focused  upon  the 
manipulation  of  visual  content  as  media  for 
training  landing  and  terrain  flight.  In  the 
first  of  these  researchers  Investigated  T-37 
pilot  landing  performance  In  response  to 
variations  of  checkerboard- like  textural  detail 
level  superimposed  upon  the  simulated  runway 
touchdown  area.  The  check  sizes  used  were  4,  8, 
15,  and  25  feet  for  four  experimental  runways. 

Two  other  runways  also  were  used;  one  a  simulated 
Air  Force  runway  with  standard  markings,  and  one 
completely  bare  except  for  a  dashed  centerline. 

A  night  runway  scene  was  also  added  bringing  the 
total  number  of  runways  to  seven.  Vertical 
velocity  at  touchdown  was  used  as  the  performance 
indicator.  Although  the  simulated  aircraft 
vertical  velocities  at  touchdown  were  much  higher 
than  those  averaged  In  actual  T-37  landings  (32 
feet  per  minute),  the  CGI  texturing  did 


significantly  reduce  vertical  velocities  at 
touchdown  in  the  simulator  ranging  from  195 
feet/minute  for  the  night  scene  to  147 
feet/minute  for  the  four-foot  texture  pattern. 

In  this  study  vertical  velocity  at  touchdown  was 
shown  to  decrease  as  a  function  of  the  amount  of 
textural  detail  available  to  the  pilot. 

FOUR  FOOT  TEXTURE  PATTERN  RUNWAY 


In  a  subsequent  study  AFHRL/OT 
Investigated  the  effects  of  three  types  of  visual 
cues:  texture  patterns  (checkerboards,  220,  440, 
or  880  feet  on  a  side),  vertical  objects  (present 
or  absent),  and  aircraft  shadow  (present  or 
absent),  upon  pilot  performance  during  low  level 
flight  in  a  simulated  A-10  aircraft.  In  this 
study  pilots  were  instructed  to  fly  50  feet  above 
the  ground  on  an  eleven-nautical  mile  course 
which  consisted  of  eight  flat  valleys  separated 
by  low  rolling  hills.  Hills  were  either  100  or 
300  feet  high.  The  pilots  were  scored  on  their 


ability  to  maintain  aircraft  altitude  at  50  feet 
plus  or  minus  30  feet  In  the  valley.  Average 
altitude  values  were  also  collected  at  the  crest 
of  each  hill,  The  pilots  flew  the  course  at  300 
knots  Indicated  airspeed  plus  or  minus  fifteen 
Knots. 

In  general  the  pilots  reported  that  all 
three  types  of  visual  cues  were  useful,  however, 
the  vertical  object  cues  and  texture  patterns 
were  of  greater  help  than  the  aircraft  shadow. 
Some  pilots  reported  that  the  aircraft  shadow  was 
particularly  useful  In  signalling  Impending 
contact  with  the  ground.  The  vertical  object 
cues,  especially,  the  tree  shaped  cones  of  known 
height  were  subjectively  very  useful  In  gauging 
height  above  ground.  The  texture  patterns  were 
also  reported  as  desirable,  but  to  a  lesser 
extent  than  the  vertical  object  cues.  Pilots 
reported  a  definite  preference  for  the  smallest 
texture  pattern  (220  feet  square)  over  the  two 
larger  size  patterns.  They  especially  disliked 
flying  over  the  largest  texture  pattern  without 
vertical  object  cues.  Pilots  would  have  also 
preferred  more  Irregular  "natural"  patterns 
rather  than  the  highly  regular  checkerboard 
features. 

440  FEET  TEXTURING  WITH  VERTICAL  OBJECTS 


Figure  2 

Both  the  texture  patterns  and  the 
vertical  object  cues  produced  statistically 
significant  differences  In  pilot  performance. 
However,  only  the  texture  pattern  cues  produced  a 
significant  effect  on  the  time  within  tolerance 
scoring  for  altitude  In  the  valleys,  and  the 
average  minimum  altitude  values  In  the  valleys. 
The  vertical  object  cues  did  have  a  significant 
effect  on  the  average  aircraft  altitude  at  the 
top  of  the  hills.  Both  the  vertical  object  cues 
and  the  texture  patterns  significantly  Influenced 
the  average  minimum  altitude  values  that  occurred 
over  each  hill.  The  presence  or  absence  of  the 
aircraft  shadow  did  not  produce  any  significant 
effects.  The  amount  of  time  (cumulative  total) 

In  the  "crashed"  condition  (in  contact  with  the 
ground)  was  low,  with  some  pilots  crashing  Into 
the  simulated  terrain  more  frequently  than 
others.  No  significant  differences  were  found 
for  this  performance  parameter  due  to  visual  cue 
variables.  Overall  In  this  study,  the  terrain 
textural  cues  appeared  to  have  a  stronger  effect 
on  pilot  performance  than  did  the  vertical  object 
cues. 

Terrain  Cues 


Data  from  the  runway  touchdown  and 
checkerboard  terrain  studies  provided  some  useful 
Insights  Into  the  problems  of  modeling  low  level 
environments.  For  one,  pilots  found  the 
checkerboard  pattern  effect  visually  monotonous, 
even  distracting.  Previous  research  (4)  and  (7) 
has  suggested  regular  pattern  texturing  Is 
Important  In  conveying  cues  to  the  observer  for 
surface  slant  orientation  and  that  Irregular 
textures  are  less  effective  In  conveying  surface 
slant  cues.  However,  these  findings  were 
relevant  to  static  rather  than  dynamic  display 
content.  Low  level  training  would  seem  to 
Involve  a  relationship  between  surface  texture 
plus  the  motion  cues  conveyed  In  simulated 
flight.  Research  with  random  texture  designs  In 
a  dynamic  display  context  suggests  motion  cues 
can  provide  for  reasonably  accurate  judgment  of 
surface  orientations  (5).  This  effect  appears  to 
be  based  on  velocity  gradient  Information  carried 
by  texture  rather  than  the  texture  gradient  per 
se  (2).  In  short,  the  motion  component  seems 
essential  as  a  cueing  element  and  there  appears 
to  be  enough  evidence  from  the  literature  and 
from  current  In-house  studies  to  warrant  testing 
the  utility  of  the  random  pattern  modeling  for 
low  level  terrain  flight  training. 

Vertical  Objects  Modeling 

Another  aspect  of  the  problem  of  low 
altitude  visual  cueing  Is  the  modeling  of 
vertical  objects.  A  continuing  problem  Is 
getting  the  maximum  number  of  cues  using  the 
least  number  of  computer  graphic  edges.  The  most, 
edge-efficient  object.  It  turns  out,  Is  a 
three-dimensional  triangle,  technically  a 
tetrahedron.  It  uses  six  edges.  In  the 
checkerboard  study,  we  used  these  shapes 
(sometimes  referred  to  a  "cones")  for  trees,  with 
the  point  up.  In  working  with  an  experimental 
CGI  combat  environment,  researchers  have  found 
the  "cones"  to  be  more  effective  as  cues  when 
turned  upside  down  so  the  broader  base  Is  more 
visible  to  the  pilot.  "Cones"  have  been  used 
very  effectively  by  pilots  to  evade  simulated 
ground  fire.  Having  the  object  point  down  seems 
to  give  a  particularly  accurate  ground  level 
cue.  We  also  found  that  planning  the  use  of 
cueing  edges  along  a  more  or  less  defined  flight 
path  Is  more  edge  efficient  since  cues  visibly 
usable  by  the  pilot  can  be  concentrated  near  his 
flight  path  rather  than  spread  over  a  large  area. 

DEVELOPING  A  CGI  LOW  LEVEL  ENVIRONMENT 

The  present  low  level  flight  CGI  was 
developed  applying  experience,  research  findings, 
and  Inferences  from  previous  CGI  developments  at 
mFHRL/OT.  Random  ground  patterns,  vertical 
object  development,  concentration  of  edges  along 
the  flight  path,  and  the  use  of  turns  In  the 
course  were  all  derived  from  previous  work. 

Other  aspects  of  modeling  were  Included  with  a 
view  toward  making  the  environment  somewhat 
realistic.  The  environment  was  modeled  to 
approach  as  nearly  as  a  2000-edge  capacity 
permits,  the  irregular  features  likely  to  be  seen 
in  actual  terrain  flight.  The  22-nautlcal  mile 
flight  path  Is  bordered  by  hills  which  slope  away 
from  It  at  realistic  rise  angles.  The  width  of 
the  corridor  ranges  from  500  to  2000  feet  and  the 
elevation  Is  0  feet  throughout.  Heading  change 
turns  through  the  course  Increase  from  23  to  45 
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to  60  to  90  degrees  In  order  of  Increasing 
difficulty  for  low  level  flight.  Inverted 
"cones"  of  several  heights  ( 25,  40,  and  55  feet) 
are  represented  with  the  shape  proportioned  as  a 
height  cue  following  recommendations  of  Stenger 
et  al  (9). 

LOU  LEVEL  TERRAIN  CGI  ENVIRONMENT 


Figure  3 

Environment  Tryout 

In  subjective  test  flight  evaluations  ten 
instructor  pilots  reported  the  random  ground 
pattern  provided  a  useful  altitude  cueing.  They 
confirmed  that  the  Inverted  "cones"  or  trees  were 
effective  as  altitude  cues  and  that  the 
peripheral  cues  provided  by  the  hills  along  the 
flight  path  were  also  effective.  Oata  from  these 
tryout  runs  was  recorded  and  analysed  for  use  In 
developing  parameters  for  an  experimental 
training  study.  The  concensus  among  pilots  was 
that  this  imagery  Is  the  most  effective  produced 
to  date  for  low  level  training  In  the  Advanced 
Simulator  for  Pilot  Training. 

SKILL  TRAINING  STUDY 

The  basic  visual  perceptual  skills  for 
low  level  flight  appear  to  be  the  hand-eye 
coordination  behaviors  involved  In  maintaining 
extremely  low  altitudes  over  a  given  terrain 
area.  Many  other  aircrew  skills  are  Involved 
Including  navigation,  systems  monitoring,  and 
communication.  But  the  basic  aircraft  handling 
skills  are  critical.  The  thrus..  of  the  present 
research  was  to  Investigate  training  techniques 
for  this  task  component. 

Various  methods  have  been  considered  for 
training  low  level  flight.  For  present  purposes 
in  the  simulator,  the  objective  was  to  train  the 
pilot  to  use  available  terrain  cues  as 
effectively  as  possible,  under  conditions  of  very 
limited  terrain  fidelity.  Long-established 
methods  of  training  visual  discrimination  have 
been  reported  by  a  number  of  researchers  (10,  11, 
6,  1,  and  3).  Relevant  visual  discriminations 
are  established  by  providing  some  form  of 
obviously  distinguishable  auxilliary  stimuli  In 
the  presence  of  the  more  subtle  discriminations 
to  be  learned.  A  two-step  flow  Is  implied: 
first,  effective  prompts  must  be  developed  and 
applied,  then  they  must  be  removed  as  the 
relevant  discriminations  are  transferred  to  the 
primary  stimuli.  To  be  effective  prompts  must 


Indeed  facilitate  relevant  discriminations,  but 
there  Is  the  possibility  that  they  may  compete 
with,  rather  than  compliment  the  primary  cues. 
Since  prompts  may  be  Initially  useful  but 
terminally  detrimental,  they  must  be  removed  as 
correct  responses  are  transferred  to 
discriminative  stimuli.  Prompt  removal  has  been 
called  fading  or  vanishing  (8)  and  Is  to  be 
accomplished  In  a  gradual,  systematic  manner. 

This  technique  has  been  used  successfully 
In  a  number  of  educational  and  pschologlcal 
contexts,  but  Its  usefulness  for  the  present 
simulator  task  has  not  been  Investigated.  In  the 
present  study  the  objective  was  to  determine  If 
prompting  would  facilitate  development  of  visual 
judgment  and  concomitant  aircraft  control  skills 
to  a  greater  degree  than  equivalent  training 
without  prompting. 

METHOD 

Subjects 

Sixteen  T-38  student  pilots  enrolled  In 
undergraduate  pilot  training  at  Williams  Air 
Force  Base  participated  as  subjects.  They  were 
all  undergoing  the  Initial  phases  of  the  T-38 
syllabus  and  none  had  received  any  form  of  low 
altitude  training  in  the  aircraft. 

Procedure 

Assignment  of  Subjects  to  Groups.  The 
subjects  were  randomly  assigned  to  one  of  two 
treatment  groups  as  follows:  The  experimental 
group  (N*8)  received  low  level  training  as 
prompted  by  lights  and  audible  tones  In  the 
cockpit.  The  prompts  were  computer  actuated  in 
response  to  specific  altitude  limits.  The 
control  group  (N«8)  received  training  Identical 
to  the  experimental  group,  less  prompting. 

Experimental  Training.  Each  subject  was 
given  a  standardized  five-minute  In-briefing 
describing  the  training  task.  The  briefing 
consisted  of  a  video  Introduction,  explanation  of 
the  flight  route,  primary  visual  references,  and 
relevant  flight  procedures. 

Subjects  assigned  to  the  experimental 
group  received  the  altitude  prompting  from  two 
small  lights  mounted  In  vertical  array  on  the 
cockpit  glareshield  and  from  audible  tones 
through  the  headset.  During  the  briefing,  they 
were  told  to  use  these  altitude  references  as  a 
means  of  attaining  consistent  low  altitude  during 
the  training.  When  the  subject  exceeded  150  feet 
above  ground  level  (AGL),  the  top  light 
illuminated  until  descent  below  that  altitude. 
When  he  descended  below  35  feet  AGL,  the  bottom 
light  illuminated  until  ascent  above  that 
altitude.  The  audible  prompts  were  presented 
simultaneously  with  the  lights;  a  1000  Hz  tone 
for  the  150- level,  and  a  600  Hz  tone  for  the 
35-foot  level.  Subjects  in  the  experimental 
group  were  told  to  use  the  35  foot  prompt 
particularly  as  a  low  level  performance  guide  and 
to  try  to  associate  the  occurence  of  the  prompt 
with  the  appearance  of  terrain  features  for  this 
altitude,  trying  to  maintain  this  altitude  as 
much  as  possible. 

Following  this  orientation,  each  subject 
was  introduced  to  the  A-10  cockpit  of  the  ASPT  by 
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an  Instructor  pilot  who  gave  him  a  standardized, 
ten-minute  familiarization  and  warmup  exercise. 
This  Included  a  takeoff,  several  turns  and  a 
landing.  He  was  also  allowed  to  "crash"  Into  the 
ground  and  "fly  through"  several  simulated  ground 
objects  in  order  to  establish  these  simulation 
effects  before  beginning  the  training  exercise. 

The  subject  was  then  Initialized  at  the 
starting  point  of  the  low  altitude  training 
environment  at  200  feet  A6L.  The  subject  was 
Instructed  to  fly  through  the  environment 
maintaining  as  low  an  altitude  as  possible 
without  crashing  into  the  ground  or  trees.  He 
was  Instructed  that  consistency  and  smoothness  of 
flight  were  Important  and  that  he  should  maintain 
an  Indicated  airspeed  of  280  to  300  knots.  He 
was  given  an  approximate  throttle  setting  as  an 
assist.  He  was  further  advised  that  turn  points 
In  the  flight  path  would  be  the  most  difficult 
segments  In  which  to  maintain  low  altitude,  and 
that  he  should  make  a  special  effort  to  stay  low 
In  the  turns  by  using  rudder.  Finally,  he  was 
Instructed  that  he  could  use  any  flying  technique 
or  ground  track  he  preferred  through  the  course 
so  long  as  he  maintained  the  minimal  altitude 
possible  without  crashing.  If  the  subject 
crashed,  he  heard  a  computer-actuated  voice  say 
"zero  altitude"  but  he  was  able  to  continue  to 
"fly  out"  of  the  crash  condition  and  complete  the 
training  trial. 

Each  subject  flew  eight  trails  over  the 
course,  with  each  trial  taking  about  4.5  minutes 
at  the  required  airspeed.  At  the  end  of  each 
trial  the  subject  was  re-lnitlallzed  at  the  same 
starting  point  and  altitude.  During  the  trials, 
no  further  verbal  instruction  or  performance 
feedback  was  provided  to  the  subject.  Time 
elapsed  for  the  entire  exercise  Including  the 
briefing  was  about  one  hour  and  ten  minutes. 

On  the  last  three  training  trials  of 
subjects  assigned  to  the  experimental  group,  the 
Intensity  of  the  light  and  tone  prompts  was  faded 
as  follows:  trial  six,  75  percent  Intensity; 
trial  seven,  50  percent;  and  trial  eight,  25 
percent.  Thus,  by  trial  nine  (the  first  test 
run),  the  prompts  had  been  completely  faded  for 
the  experimental  group. 

Following  completion  of  eight  consecutive 
training  trials,  all  subjects  were  given  two 
additional  trials  as  a  test  of  training 
effectiveness.  During  the  two  test  runs  the 
number  of  vertical  objects  (trees)  in  the  CGI 
scene  was  reduced  by  50  percent.  Subjects  were 
also  Instructed  to  maintain  a  more  critical 
airspeed  tolerance  (300  KIAS,  plus  or  minus  5 
knots) . 

Performance  Measures 


Dependent  measures  for  the  simulated  low 
level  task  were:  mean  altitude,  mean  vertical 
velocity,  and  frequency  of  crashes  during 
training  trials  and  test  runs.  The  altitude 
measure  and  crash  frequency  were  taken  as 
indicators  of  the  subject's  ability  to  use 
available  visual  cues  to  maintain  minimally  low 
level  safe  flight.  Vertical  velocity  measures 
were  intended  as  an  Indicator  of  aircraft  control 
and  overall  smoothness  of  flight.  Measurements 
of  these  parameters  were  taken  during  the  three 


most  difficult  turns  (45,  60,  and  90-degree 
heading  changes).  The  start  and  stop  of  turn 
maneuvers  for  each  subject  on  each  trial  were 
determined  at  the  point  where  bank  angle  exceeded 
(start)  and  dropped  below  (stop)  15  degrees 
nearest  the  geographical  x-y  coordinates  of 
turns.  The  measurement  of  a  mean  altitude  during 
wings  level  flight  was  also  taken  as  a  general 
Indicator  of  the  altitude  attained  over  the  route 
for  each  trial.  This  was  the  residual  of 
altitude  sampling  by  the  system  across  the  entire 
flight  course,  less  the  turns  and  hills  (150  and 
200  feet  high)  placed  at  two  points  across  the 
flight  path.  Performance  measures  were  sampled 
at  a  rate  of  30  Hz  during  all  training  and 
testing  for  each  subject  via  a  computerized  data 
record  system.  Following  data  collection  these 
data  were  re-sampled  at  a  one  Hz  rate  for 
reduction  and  analysis. 

Experimental  Design 

A  Lindquist  type  1  experimental  design 
was  used.  One-way  analysis  of  variance  was 
performed  for  altitude,  vertical  velocity,  and 
crash  frequency  data  to  test  treatment  by  subject 
by  trials  effects  for  each  of  these  performance 
parameters. 


RESULTS 

Figures  4-7  show  the  mean  altitude  for 
each  of  the  eight  training  trials  for  the 
experimental  and  control  groups.  Between  group 
differences  were  found  statistically  significant 
across  mean  altitudes  on  both  the  90-degree  turn 
(F*  5.82,  p  <.03)  and  the  45-degree  turn 
( F=13 . 25 ,  p <.005 )  trials  as  Illustrated  in 
Figures  7  and  5  with  the  experimental  group 
achieving  the  lower  altitudes  across  trials. 
Although  the  experimental  group's  achieved  trial 
means  during  wings  level  flight  and  the  60-degree 
turn  were  also  numerically  lower  than  the  control 
group  (Figures  4  and  6),  these  differences  are 
not  statistically  significant. 
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groups  as  a  result  of  the  low  level  training. 
Learning  curves  for  mean  altitude  across  the 
eight  training  trials  (combined  group  trial 
effects)  were  significant  on  all  turns  (45-degree 
-  F»9.65,  p<.0001;  60-degree  -  F«10.93,  p< 
.0001;  90-degree  -  F»8.21,  p^.OOOl). 
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Achieved  altitude  performance  during  the 
testing  concltlon  Is  also  shown  <in  each  of  the 
figures  as  a  two-trial  mean  plotted  for  trials 
nine  and  ten.  None  of  these  between  group 
comparisons  was  statistically  significant. 

ANOVA  comparisons  of  vertical  velocity 
performance  on  all  task  segments  for  all  trials 
revealed  no  significant  findings.  Frequency  of 
crashes  during  the  training  trials  did  show  a 
group  effect.  The  control  group  subjects  crashed 
significantly  (F«5.43  p  <.03)  fewer  times  (.94 
crashes  per  trial)  than  did  the  experimental 
group  (1.86  crashes  per  trial).  However,  no 
reliable  difference  between  the  groups'  crash 
performance  was  found  during  the  test  runs. 

Aside  from  the  treatment  effects,  the 
trials  effects  revealed  by  the  ANOVAs  for 
altitude  during  the  training  show  consistent  and 
highly  significant  practice  effects  for  both 
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DISCUSSION 

The  purpose  of  this  study  was  to  assess 
the  utility  of  a  specific  type  of  augmented 
feedback  (audiovisual  prompting)  for  training  low 
altitude  perceptual  motor  skills.  The  use  of  the 
prompts  did  enable  subjects  in  the  experimental 
group  to  achieve  significantly  lower  altitude 
perfomance  on  two  of  the  four  critical  task 
segments  during  the  training  trials.  While  in 
evidence,  this  effect  was  Insufficiently  powerful 
to  produce  reliable  performance  Improvements  over 
the  control  group  during  the  test  runs.  This 
inadequacy  appears  to  be  a  problem  specific  to 
the  prompting  technique  and  not  the 
discriminative  stimuli  available  to  the  subject 
via  the  computer  imagery. 

The  CGI  environment  was  effective.  The 
highly  reliable  trials  effect  indicates  that  the 
visual  Imagery  was  Indeed  powerful  In  conveying 
altitude-relevant  discrimination  cues  to  the 
pilot  and  is  also  consistent  with  the  plaudits 
this  visual  environment  has  received  from  a 
considerable  number  of  pilots  experienced  In 
terrain  flight. 

While  vertical  velocity  data  provided  no 
additional  clues  to  performance  differences 
between  the  groups,  the  crash  frequency  data 
present  something  of  a  puzzle.  While  it  would 
seem  that  the  35-feet  prompts  could  serve  as  a 
reasonably  effective  warning  away  from  the  ground 
during  the  training  trials,  the  data  Indicate 
otherwise.  Prompts  seem  to  have  interf erred  In 
some  way  with  the  subjects  aircraft  control, 
perhaps  distracting  them  below  35  feet.  If  this 
is  the  case,  this  type  of  prompting  Is  obviously 
inappropriate  and  dangerous  for  the  task. 
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However,  the  specific  process  operable  within  the 
prompting,  for  the  present,  can  only  be 
speculated  upon. 

Returning  to  theory  momentarily,  the  use 
of  effective  prompting  presupposes  a  two-stop 
flow:  (1)  the  arrangement  of  suitable  ancillary 
Information  by  which  transfer  of  desired  behavior 
to  relevant  stimuli  can  be  facilitated,  and  (2) 
an  effective  means  for  removing  the  prompts  once 
the  desired  responses  are  established  so  that  the 
learner  no  longer  relies  upon  the  ancillary 
Information.  The  technique  can  be  problematic. 
Prompts  are  actually  additional  Information  to  an 
array  of  existing  complex  stimuli.  As  a 
medlat.lnnal  device,  prompts  must  be  sufficiently 
powerful  to  Justify  the  additional  Information 
load.  However,  If  prompts  become  too  obtrusive, 
primary  task -re levant  stimuli  may  be 
over-shadowed  precluding  desired  assoclatlonal 
transfer,  and  defeating  the  objective  of 
prompting. 

It  Is  not  clear  from  the  results  or  the 
above  theory  why  the  present  prompting 
application  produced  less  than  a  useful  level  of 
training.  On  the  one  hand  they  seemed  too  weak 
during  trial's  to  produce  strongly  differential 
training  effects,  at  least  for  the  altitude 
performance  dimension.  Conversely,  as  terrain 
avoidance  cues  (crash  data).  It  seems  the  prompts 
were  too  obtrusive  or  In  some  other  way 
Inappropriate,  to  the  point  of  possible 
performance  interference  at  extremely  low 
altitude.  Aside  from  the  training  trial  effects, 
training  transfer  to  the  test  runs  shows  no 
differences  in  group  performance  for  either 
altitude  or  crash  freauency,  the  test  runs  being 
the  crucial  factor  in  the  present  effectiveness 
comparisons. 

Perhaps  variation  of  the  prompting  would 
significantly  improve  effectiveness.  It  is 
possible  that  the  altitude  limits  set  for  tne 
present  study  were  not  appropriate  for  the 
subject  population,  although  the  limits  were 
arrived  at  systematically  as  a  result  of  repeated 
trials  by  T-38  and  research  instructor  pilots. 
Perhaps  too,  a  more  flexible  or  adaptive  system 
of  prompting  in  which  the  altitude  limits  of 
prompts  vary  as  a  function  of  student  performance 
across  trials  would  be  effective.  This  Is  ver. 
speculative,  and  on  the  basis  of  present 
evidence,  it  would  seem  hard  to  justify  the  time 
and  costs  of  developing  such  a  prompting  system. 
Results  of  the  present  study  do  not  appear  to 
warrant  continued  investigation  of  this  technique 
for  training  low  level  flying  skills. 

Other  substantive  questions  remain 
relative  to  training  terrain  flight  In  simulators 
which  deserve  investigation  as  research  Issues. 
Visual  environment  issues  include  influence  of 
field  of  view  upon  low  level  training  and 
techniques  for  Improved  modeling  of  object  and 
texturing  features.  Training  variables  Include 
Investigation  of  task  difficulty  and  task 
sequencing  variables,  and  alternative  performance 
feedback  techniques  for  terrain  flight  training, 
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ABSTRACT 

Visual  systems  with  a  single  window  display  are  often  utilized  in  ground  based  simulators 
used  to  study  helicopter  flying  qualities  during  visual  low  altitude  maneuvering  tasks.  The 
effects  of  this  limited  field  of  view  (FOV)  on  pilot  assessments  of  flying  qualities  are  uncer¬ 
tain.  A  study  was  conducted  using  a  variable  stability  I'H-IH  helicopter  to  compare  restricted 
and  unrestricted  FOV  for  a  range  of  flying  qualities.  With  the  restricted  FOV,  the  pilots 
reported  reduced  ground  track  precision  owing  to  loss  of  visual  contact  with  the  course  markers. 
However,  the  predictable  ground  track  of  the  repeated  S-turn  task  with  no  obstacles  made  it  easy 
for  them  to  anticipate  their  maneuvers,  and  resulted  in  only  a  slight  degradation  of  pilot  rat¬ 
ings.  This  degradation  was  not  sensitive  to  large  changes  in  helicopter  flying  qualities. 


INTRODUCTION 

Factors  that  can  affect  the  fidelity  of  ground 
based  simulation  include  the  mathematical,  model, 
visual  system,  motion  system,  aural  cueing,  cockpit 
layout,  and  environmental  conditions.  Visual  sys¬ 
tems  with  a  single  window  display  are  often  utilized 
in  the  ground  based  simulators  used  to  study  heli¬ 
copter  flying  qualities.  The  purpose  of  this  experi¬ 
ment  was  to  evaluate  the  effects  of  FOV  restrictions 
on  the  pilot's  perception  of  flying  qualities  for  a 
low  altitude  maneuvering  task.  The  study  was  con¬ 
ducted  using  a  variable  stability  helicopter,  with 
the  evaluation  pilot's  FOV  restricted  to  that  of  a 
single  window  simulator  display.  The  procedures 
described  in  (1)  and  (2)  were  used  to  restrict  the 
field  of  view.  To  determine  the  sensitivi ty  of  FOV 
effects  to  large  changes  in  handling  qualities, 
modified  helicopter  flying  qualities  configurations 
were  selected  from  those  used  in  the  experlmect 
reported  in  (3);  they  are  «v.tranarlzed  in  Table  I. 

The  four  FOV  configurations  used  in  this  experiment 
are  defined  in  Table  2. 


EXPERIMENTAL  APPROACH 
FOV  Setup  Procedures 

The  simulator  FOV  with  respect  to  pilot  design 
eyepoint  was  measured  using  the  specially  adapted 
transit  shown  in  Figure  1.  This  FOV  was  then  mapped 
onto  the  test  helicopter  windscreen,  as  shown  in 
Figure  2.  Orange  masking  film  was  Installed  on  the 
inside  of  the  aircraft  windscreen,  except  for  the 
mapped  area.  When  viewed  through  the  blue  visor 
(Fig.  3)  all  sections  of  the  aircraft  windscreen 
covered  by  the  orange  masking  film  become  opaque. 
Objects  viewed  through  the  mapped  window  appeared  to 
the  pilot  as  different  shades  of  gray.  The  light 
transmission  characteristics  of  the  orange  masking 
film  and  blue  visor  are  presented  in  Figure  4.  In 
the  right  seat,  the  safety  pilot  had  only  a  small 
loss  of  visual  acuity  looking  through  the  orange 
masking  film.  On  bright,  sunny  days,  the  safety 
pilot  used  the  sun  visor  on  his  helmet  to  reduce  the 
glare  from  the  orange  masking  film.  No  flying  was 
conducted  under  IMC  or  at  night  because  of  safety 
considerations. 

Evaluation  Task 

The  FOV  flight  experiment  was  conducted  at  Ames 
Research  Center's  flight  research  facility  at  Crow's 
Landing  in  conjunction  with  the  flying  qualities 


experiment  described  in  (3).  The  task  consisted  of 
low  altitude  maneuvering  around  a  series  of 
markers,  set  up  along  ar;  8,000  ft  runway  (Fig.  5). 
The  pilots  were  instructed  to  traverse  the  course 
while  using  the  1,000  ft  runway  markers  as  points 
around  which  to  turn  the  helicopter  fuselage  with¬ 
out  considering  rotor  blade  clearance.  The  pilots 
were  also  instructed  to  maintain  an  airspeed  of 
60  knots  and,  for  safety,  an  altitude  of  100  ft- 

Data  Acquisition 

Quantitative  flight  data  were  recorded  by  an 
on  board  analog  magnetic  tape  recorder  and  were 
also  telemetered  to  a  ground  station  for  real  time 
monitoring  and  postflight  analysis.  Variables 
recorded  included  control  positions,  attitudes, 
rates,  accelerations,  airspeed,  and  altitude.  A 
tracking  radar  produced  ground  track  data  for  real 
time  x-y  plots.  For  each  FOV  configuration,  the 
pilots  provided  an  overall  Cooper-Harper  handling 
qualities  rating  and  specific  commentary  on  the 
precision  of  control  through  the  course.  A  Cooper- 
Harper  rating  scale  is  shown  in  Figure  6. 


RESULTS  AND  DISCUSSION 

The  results  of  this  experiment  are  presented 
in  the  form  of  qualitative  pilot  opinion  ratings 
and  comments  (Table  3)  and  quantitative  flight  data 
(Table  4  and  Figures  8  and  9). 

Pilot  Ratings  and  Commentary 

For  the  task  used  in  this  experiment,  the  pilot 
ratings  were  not,  in  general,  sensitive  to  restric¬ 
tions  in  FOV.  This  was  true  for  large  changes  in 
aircraft  flying  qualities.  The  best  flying  quali¬ 
ties  configuration,  UH-1H  manual  mode  (MAN),  with 
no  FOV  restrictions  was  given  a  handling  qualities 
rating  (HQR)  of  3,  and  the  worst  flying  qualities 
configuration  (R8.5,  defined  as  sluggish  and  highly 
coupled)  was  rated  about  7.5.  With  these  same  fly¬ 
ing  qualities  configurations,  the  restricted  FOV 
did  not  produce  degradations  greater  than  1  HQR  for 
pilots  familiar  with  the  task.  Only  pilot  R,  who 
initially  was  not  familiar  with  the  task,  rated  the 
good  (MAN)  configuration  at  5  on  his  initial  run 
with  a  restricted  FOV.  The  rating  was  lowered  to 
4.5  and  4  for  the  second  and  third  runs,  respec¬ 
tively.  However,  these  ratings,  which  reflect  pilot 
learning,  did  not  converge  to  the  3  rating  that  the 
pilot  gave  the  nonrestricted  FOV  case.  Pilot  D 
reported  a  maximum  degradation  of  1  HQR  with  the 
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restricted  FOV  on  the  Initial  qualitative  checkout 
flight.  Subsequent  flights  by  pilots  D,  G,  M,  and 
T  failed  to  produce  greater  than  a  one-half  HQR 
degradation,  as  shown  in  Figure  ?.  On  his  first 
flight,  pilot  M  actually  rated  the  restricted  FOV 
case  better  than  the  unrestricted  case  for  the 
highly  damped,  highly  coupled  R8. 5  configuration,  as 
shown  in  Figure  7  and  Table  3.  However,  on  his 
second  flight,  he  rated  both  restricted  and  unre¬ 
stricted  FOV  the  same  for  configuration  R8.5.  The 
fact  that  all  pilots  except  pilot  R  were  familiar 
with  the  task  from  the  experiment  reported  in  ref¬ 
erence  3,  and  that  the  flight  course  was  predictable, 
may  have  Influenced  these  evaluations.  Pilot  T 
noted  that  it  required  only  one  pass  through  the 
course  to  be  able  to  guess  where  the  next  marker  was 
located,  and  that  a  more  difficult  course,  tighter 
turns,  or  a  less  predictable  flightpath  might  have 
shown  a  greater  effect  due  to  limiting  the  field  of 
view. 

Control  Activity  Statistics 

A  statistical  summary  of  lateral  and  longitudi¬ 
nal  control  positions,  series-servo  positions,  air¬ 
craft  rates,  and  attitudes  for  selected  data  runs  is 
presented  in  Table  4.  For  the  good  handling  qual¬ 
ities  configuration  (SBO*)  the  data  show  very  little 
difference  between  the  restricted  and  unrestricted 
FOV  cases.  For  the  poor  handling  qualities  config¬ 
uration  (R8.5),  the  restricted  FOV  resulted  in  a 
small  reduction  in  standard  deviations,  especially 
for  the  longitudinal  parameters. 

Aircraft  Flightpath  Control 

The  moderately  high  altitude  (100  ft)  at  which 
the  task  was  flown  meant  that  precise  aircraft 
flightpath  cues  were  not  available.  In  addition, 
the  pilots  reported  losing  ground  track  precision 
because  of  loss  of  visual  contact  with  the  course 
markers  during  runs  with  the  restricted  FOV.  How¬ 
ever,  most  portions  of  the  radar  derived  ground 
track  plots  shown  in  Figures  8  and  9  indicate  a 
relatively  close  correlation  in  ground  track  between 
restricted  and  unrestricted  FOV  runs.  This  may 
imply  that  the  predictable  course  layout  of  repeated 
S-tums  with  no  obstacles  made  it  easy  for  the 
pilots  to  anticipate  their  maneuvers. 

The  pilots  also  experienced  difficulty  in  judg¬ 
ing  aircraft  height  above  the  ground  with  the 
restricted  FOV  without  referring  to  the  radar  altim¬ 
eter.  This  was  attributed  to  the  lack  of  peripheral 
cues  with  the  restricted  FOV  and  the  high  reference 
altitude  above  ground  level. 

Related  Flight  Program 

Previous  unpublished  restricted  FOV  flight 
tests  performed  at  Ames  Research  Center  in  December 
1973  using  a  UH-1B  helicopter  produced  similar  qual¬ 
itative  results  for  low  level  day  flying.  Those 
testa  consisted  of  performing  a  numoer  of  basic  low 
level  helicopter  maneuvers,  including  takeoff,  land¬ 
ing,  precision  hover,  pedal  turns,  and  air  taxi 
tasks;  the  maneuvers  were  performed  over  a  runway 
with  no  obstacles  or  traffic  to  monitor.  Maneuvers 
were  performed  with  no  FOV  restrictions,  with  a  hel¬ 
met  fitted  with  a  visor  that  restricted  the  pilot's 
FOV  to  48°  :<  34°,  and  also  with  opaque  material  used 
to  mask  off  the  windscreen  to  produce  a  48°  x  34° 

FOV.  A  slight  degradation  in  performance  was 
reported  for  the  case  with  the  visor.  This  was 
attributed  to  the  fact  that  inadvertent  head  move¬ 


ment  could  be  interpreted  as  an  attitude  change  of 
the  aircraft  causing  the  pilot  to  make  undesirable 
control  Inputs.  With  the  masked  windscreen,  the 
pilot  was  able  to  maintain  a  performance  level  com¬ 
parable  with  the  unrestricted  FOV  case. 

FOV  Requirements  for  Hap-of-the-Earth  Flight 

The  FOV  effects  on  flying  qualities  ratings  for 
the  task  used  in  this  experiment  may  not  apply  to  a 
course  requiring  the  pilot  to  fly  around  real 
obstacles  or  to  actual  Nap-of-the- Earth  (NOB) 
flight.  With  the  restricted  FOV,  the  pilot  would  be 
unable  to  monitor  rotor  blade  clearance  in  confined 
areas  or  monitor  flightpath  during  sideward  or  rear¬ 
ward  flight.  Reference  4  reported  the  operational 
suitability  evaluation  of  the  UH-60A  helicopter  in 
an  advanced  attack  helicopter  vole  which  involved 
low  level,  contour  flight  and  NOE  flight.  The  moot 
critical  reduction  in  visibility  in  the  utility 
helicopter  compared  with  that  in  the  attack  helicop¬ 
ter  was  the  pilots'  inability  to  see  the  main  rotor 
at  the  90°  point.  During  the  evaluation,  this 
accounted  for  six  blade  strikes  by  the  utility  heli¬ 
copter  crew;  there  were  no  blade  strikes  by  the 
attack  helicopter  crew.  Other  limitations  of  the 
utility  helicopter  that  were  noted  included  limita¬ 
tions  in  overhead  visibility  and  downward  visibility 
both  to  the  left  and  right. 

Attempts  to  establish  a  realistic  NOF.  task  for 
the  experiment  reported  in  (3)  and  the  experiment 
reported  herein  were  not  successful.  Safety  precau¬ 
tions  required  that  a  minimum  altitude  of  100  ft  be 
used.  The  necessity  to  monitoi  both  the  aircraft 
and  real  time  telemetered  data  required  that  the 
experiment  be  conducted  in  the  immediate  vicinity  of 
the  Crow's  Landing  data  station.  Installation  of 
high  obstacles  to  fly  around  was  precluded  because 
they  would  have  interfered  with  other  air  traffic  at 
Crow's  landing. 


CONCLUSIONS 

The  flight  experiment  described  in  this  paper 
was  conducted  to  determine  the  effects  of  field  of 
view  (FOV)  on  helicopter  flying  qualities  when  a 
low  altitude  maneuvering  task  was  performed.  The 
task  consisted  of  flying  "S-tums"  over  a  series  of 
runway  markers,  separated  1,000  ft  longitudinally 
and  250  ft  laterally,  at  an  airspeed  of  60  knots 
and  at  an  altitude  of  100  ft.  Qualitative  and  quan¬ 
titative  results  were  obtained  for  the  nominal  FOV 
of  a  UH-1H  helicopter  and  with  the  FOV  restricted 
to  that  of  a  simulator  with  a  single  window  visual 
system,  FOV  effects  were  evaluated  on  the  basic 
test  helicopter  and  on  a  variety  of  configurations 
with  degradud  flying  qualities.  From  the  limited 
data  obtained  during  this  experiment  and  from 
related  data  discussed  in  this  paper,  the  following 
trends  and  conclusions  ate  notea: 

1.  The  results  showed  only  a  minimal  variation 
in  pilot  flying  qualities  ratings  and  statistical 
data  while  performing  this  task  with  restricted  and 
unrestricted  FOV.  Restricting  the  FOV  resulted  in 

a  maximum  degradation  of  one  pilot  rating  even  for 
configurations  with  widely  differing  flying 
qualities. 

2.  The  predictable  ground  track  of  the 
repeated  S-turn  course,  the  relatively  high  alti¬ 
tude,  and  lack  of  obstacles,  made  it  easy  for  the 
pilots  to  artlcipate  their  maneuvers  and  did  net 
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demand  high  precision.  These  factors  appeared  to 
lessen  the  effect  of  reduced  field  of  view. 

3.  The  restricted  FOV  made  height  control 
difficult  and  required  a  frequent  scan  of  the  radar 
altimeter  to  maintain  the  reference  altitude. 
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TABLE  1.  EXPERIMENT  FLYING  QUALITIES 
CONFIGURATIONS  _ __ 


Configuration 

Description 

MAN 

Manual  mode  or  basic  UH-1H  helicopter 

SBO* 

Manual  mode  using  tne  series  servos 

In  the  automatic  flight  control  sys¬ 
tem.  Approximately  same  character¬ 
istics  as  basic  helicopter  with 
increased  yaw  damping,  I!r  - -3. 5  sec-1 

R4 

Roll  damping  (L_)  ■  -4  sec-1 

Cross  coupling  (Lq/Lp) ,  (Hp/Mq)  -  0 

R4.5 

Roll  damping  (Lp)  «  -4  sec-1 

Cross  coupling  (Lq/Lp),  (Mp/Mq)  -  0.5 

R8 

Roll  damping  (Lp)  ■  -C  sec-1 

Cross  coupUng  (Lq/Lp),  (Mp/Mq)  -  0 

R8.5 

Roll  damping  (Lp)  “  -8  sec-1 

Cross  coupling  (Lq'Lp),  (Mp/Mq)  -0.5 

Note:  During  the  experiment,  the  lateral  con¬ 
trol  sensitivity  (L$)  was  held  at  0.55  rad/sec2/in. , 
and  the  longitudinal  control  sensitivity  (M$)  was 
held  at  0.14  rad/sec2/ln. 


TABLE 

2.  EXPERIMENT  FIELD  OF  VX2W 
CONFIGURATIONS 

Configuration 

Description 

1 

No  FOV  restrictions 

2 

Pilot  wearing  blue  visor,  no  orange 
masking  film  on  windscreen 

3 

Helicopter  windscreen  covered  by 
orange  masking  film  except  for  a 
clear  48*  x  36*  area  in  front  of  the 
evaluation  pilot;  no  blue  viaor 

4 

Pilot  wearing  blue  visor,  orange 
masking  film  on  windscreen 
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TABLE  3.  CONCLUDED 


Configuration? 

Pilot 

Helicopter 

Visual 

rating 

SBO* 

3 

T 

3 

4 

T 

3.5 

Visual  acuity  good,  everything  clear;  only  takes  one  pass  through  course 
even  when  you  cannot  see  markers  to  guess  where  next  one  will  be;  a  more 
difficult  course  or  tighter  turns  or  a  less  predictable  flightpath  might 
show  up  in  worst  ratings 

R4 

3 

T» 

3.5 

4 

T 

3.5 

Very  little  difference  in  running  course  with  restricted  FOV 

K4.5 

*1 

T 

4.5 

4 

T 

4.5 

Airspeed  control  more  difficult;  cannot  tell  airspeed  by  looking  outside 

R8 

3 

T 

4 

4 

T 

4 

R8.5 

3 

T 

7 

4 

T 

5 

SBO* 

1 

D 

3+ 

3 

D 

3.5 

4 

D 

3.5 

Can  see  enough  going  out  of  a  turn  or  Into  a  turn  to  complete  course 

R8 

i 

D 

5' 

3 

P 

5 

Definite  degradation  in  flying  qualities 

4 

D 

5 

R8.5 

1 

D 

7+ 

3 

D 

7 

4 

D 

7 

Do  not  think  FOV  degrades  this  configuration  much;  biggest  problem  is  the 
almost  sustained  oscillation  of  airframe 

SBO* 

1 

M 

3.5+ 

1 

M 

4 

Roll  sensitivity  felt  high;  gained  40  ft  but  was  able  to  correct;  air¬ 
speed  control  was  good 

SBO* 

3 

M 

4 

Used  the  clear  area  of  windscreen  mostly;  problem  with  airspeed  not  FOV 

SBO* 

4 

M 

4 

Hsu  to  use  airspeed  indicator  due  to  lack  of  peripheral  cues 

R8.5 

1 

M 

7+ 

1 

M 

7.5-8 

Configuration  has  loping  motion;  cannot  establish  precise  bank  angle, 
altitude  or  airspeed 

R8.5 

1 

M 

7.5 

Large  control  inputs  required;  problems  with  airspeed,  altitude,  roll 
sensitivity,  and  damping 

R8.5 

3 

M 

7.5 

R8.5 

4 

M 

7.5-8 

Very  uncomfortable 

+ Averaged  data  from  (3). 
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TABLE  4.  STANDARD  DEVIATION  SUMMARY  FOR  SELECTED  LATERAL  AND  LONGITUDINAL  PARAMETERS 


5 


60, 


I 

i 

»- 


28 
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BO 


RESEARCH  PILOT  EYEPOINT  (LEFT  SEAT) 
48.4  In.  FUSELAGE  STATION 
21.0  BUTT  LINE 
64  2  WATERLINE 


UH-IH  HESF.ARCH  PILOT 

FRONT  WINDOW  UH-IH  SAFETY  P'LOT 


- 1 - 1  i - 1 - 1 - 1.  J.  ■  J 

100  75  60  2B  0  26  60  75 
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Figure  2.  UH-IH  Helicopter  and  S19  Simulator  Vision  Plot 


BLUE  VISOR 

ORANGE  MASKING  FILM ' 


Figure  3.  Blue  Visor  and  Orange  Masking  Film 


ORANGE  MASKING  FILM 
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WAVELENGTH,  nm 

Figure  4.  Light  Transmission  Characteristics 
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Al  RCA  APT 
CHARACTERISTICS 

DIMAN  Dt  ON  TNI  PILOT  IN  SILICTCD 
TASK  OR  RIOUIRCD  OPlRATtON*  1 

Excellent 

Highly  detirable 

Pilot  compentation  not  a  factor  for 
desired  performance 

Sood 

Negligible  dtf  icitnciet 

Pilot  computation  not  •  fset'or^or 
ditired  performance 

Fair-Some  mildly 
unpleasant  datlctgncto 

Minimal  pilot  compensation  required  for 
detlreti  performance 

ADEQUACY  POA  SELECTED  TASK  OA 
REQUIRED  OPERATION* 


PILOT 


Deficiencies 
►  warrant 

Minor  but  annoying 
daficlenclet 

b nired  performance  require!  moderate 
pilot  compentation 

Moderately  objectionable 
deficiencies 

Adequate  performance  require! 
contiderebie  pilot  compentation 

Vary  objectionable  but 
tolerable  deficlenciet 

Adequate  performance  require!  ex  tan  live 
pilot  compentation 

adequate 

/  performance 
<  attainable  with  a 
^  tolerable  pitot 
-  .workload?, 


Major  daficlanciat 

Adequate  performance  net  attainable  with 
maximum  tolerable  pilot  compentation. 
Controllability  not  in  question. 

Major  daficlenciet 

Contiderabla  pilot  compensation  it  required 
for  control 

Major  deficitncifei 

Intente  pilot  compentation  it  required  to 
retain  control 

|  Major  deficitnciet 

Control  w'll  be  lott  during  tome  portion  of 
required  operation 

Cooper  Harper  Rtf.  NASA  TND-B  '53 


*  Definition  of  required  operation  intohm  designation  of  flight  phtM  eod/o 
uibphatei  with  accompanying  condition*. 


DEFINITIONS  FROM  TN-D-5153 


COMPENSATION 

The  measure  of  additional  pilot  effort 
and  attention  required  to  maintain  a 
given  level  of  performance  in  the  face  of 
deficient  vehicle  characteristics. 

HANDLING  QUALITIES 

Those  qualities  or  characteristics  of  an 
aircraft  that  govern  the  ease  and  preci¬ 
sion  with  which  a  pilot  is  able  to  perform 
the  tasks  required  in  support  of  an  air¬ 
craft  role. 

MISSION 

The  composite  of  pilot-vehicle  functions 
that  must  be  performed  to  fulfill  opera¬ 
tional  requirements.  May  be  specified  for 
a  role,  complete  flight,  flight  phase,  or 
flight  subphase. 


PERFORMANCE 

The  precision  of  control  with  respect  to 
aircraft  movement  that  a  pilot  is  able  to 
achieve  in  performing  a  task.  ( Pilot - 
vehicle  performance  is  a  measure  of 
handling  performance.  Pilot  perform¬ 
ance  is  a  measure  of  the  manner  or 
efficiency  with  which  a  pilot  moves  the 
principal  controls  in  performing  a  task.) 

ROLE 

The  function  or  purpose  that  defines  the 
primary  use  of  an  aircraft. 

TASK 

The  actual  work  assigned  a  pilot  to  be 
performed  in  completion  of  or  as  repre¬ 
sentative  of  a  designated  flight  segment. 


WORKLOAD 

The  integrated  physical  and  mental  effort  required 
to  perform  a  specified  piloting  task. 


Figure  6.  Handling  Qualities  Rating  Scale 
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Figure  7. 
Ratings 


Effect  of  Visual  Configuration  on  Pilot 
for  Selected  Aircraft  Configurations 


Figure  8.  Ground  Track  Plot  Showing  Effects  of  FOV 
for  Configuration  SBO*:  Pilot  D 


Figure  9.  Ground  Track  Plot  Showing  Effects  of  FOV  for  Configuration  SBO*:  Pilot  M 
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ABSTRACT 


The  resolution  capabilities  of  computer  Image  generators  (CIG)  used  for  simulation 
and  training  have  advanced  to  the  degree  that  they  exceed  the  capabilities  of  existing 
shadow  mask,  direct  view  color  TV  displays  and  color  TV  projectors.  One  solution  to  this 
problem  Is  the  modern  day  Implementation  of  a  trinoscope  color  display  that  uses  the 
optical  merging  of  three  high  resolution  monochrome  cathode  ray  tubes— red,  green,  blue 
color  phosphors,  respectively— to  produce  a  full  color  Image.  Such  systems  are 
particularly  suitable  for  telescopes  and  periscopes  needed  In  tank  or  submarine 
simulations  where  the  color-combining  optics  can  be  Integrated  Into  the  simulated  sight 
optics.  This  paper  describes  the  technical  advances  required  to  assure  maximum  resolution 
and,  more  Importantly,  superior  color  convergence  (l.e.,  the  capability  to  make  the  three 
CRT  Images  fall  exactly  on  top  of  each  other  so  that  the  resulting  full  color  image  is 
produced  without  undesirable  color  fringing). 


INTRODUCTION 

There  are  many  training  situations  where  It  Is 
desirable  to  view  a  color  TV  Image  through  the 
eyepiece  of  a  simulated  telescope  or  periscope. 
Examples  would  be  the  gunner's  and  commander's 
sights  of  a  modern  main  battle  tank  or  the 
periscope  of  a  killer  submarine.  These  situations 
usually  carry  with  them  a  fairly  stringent 
constraint  on  the  total  volume  available  for  the 
display  and  optics.  This  means  that  the  size  of 
the  display  as  well  as  the  size  of  Its  Image  must 
be  minimized  while  still  meeting  the  criteria  of 
color,  resolution,  etc.  A  minimal  Image  size 
likewise  conserves  on  optical  path  length,  which 
further  reduces  display  system  size.  These 
constraints  produce  requirements  for  an  Image 
source  which  are  far  beyond  those  of  the 
conventional  shadow  mask  CRT.  The  one  Image  source 
which  provides  the  highest  TV  resolution  can  also 
provide  full  color  In  a  very  small  raster  format. 
It  Is  the  trl noscope. 

The  Optical  Industry  and  Systems  Encyclopedia 
and  Dictionary  defines  "trinoscope"  as  "a  color- 
television  viewing  system  with  three  kinescopes, 
three  lenses,  and  three  deflection  yokes  used  to 
form  the  red,  green,  and  blue  Images  required  for  a 
tricolor  television  projection."  Kinescope  Is 
another  name  for  a  cathode  ray  tube  (CRT).  The 
only  disagreements  with  the  above  definition  are 
the  use  of  the  words,  "three  lenses"  and 
’projection."  Various  combinations  of  lenses, 
mirrors,  and  dlchrolc  filters  can  be  used-each 
with  Its  own  sources  of  problems  and  each  with  Its 
advantages.  Also,  as  will  be  seen  In  Figures  5  and 
6  later,  the  system  can  be  direct  view.  Figures  1 
through  4  show  various  configurations  that  have 
been  used  in  the  past  and  which  are  now  being 
revived  for  home  color  TV  projection  systems. 
Figures  1  and  2  show  configurations  which  meet  the 
original  definition.  In  Figure  1,  the  Images  of 
the  viewing  plane  are  tipped  and  must  be  held 
within  the  system  depth  of  focus  to  provide 
acceptable  image  quality.  This  problem  can  be 
resolved  somewhat  by  applying  the  Schelmpflug 


condition,  commonly  used  with  view  cameras  and  In 
photogrammetry. (1)  Electronic  keystone  correction 
Is  required.  Within  the  Figure  2  configuration, 
the  lens  axes  are  all  parallel  and  perpendicular 
to  the  Image  plane,  but  the  outer  CRTs  are 
displaced  laterally  outward  so  that  the  outer 
optical  axes  converge  at  the  screen.  This 
combination  reduces  the  need  for  keystoning 
correction  and  provides  better  depth  of  field 
control  but  places  more  constraints  on  the  lens. 

Figure  3  Illustrates  a  trinoscope  system  that 
uses  only  two  lenses  and  Is  used  commercially  for  a 
home  TV  projector.  Figure  4  uses  only  one  lens  and 
Is  the  basis  of  ESP's  Aquavlslon  projector.  Even 
though  this  system  does  have  a  single  exit  pupil --a 
necessary  feature  for  a  telescope/periscope  trailer  ^ 
application— the  crossed  dlchrolcs  become  difficult 
to  implement  for  higher  resolution  systems.  The 
discontinuity  at  the  Intersection  of  the  four 
dlchrolc  mirror  elements  acts  as  a  distributed 
central  obscuration,  In  optical  terms,  limiting 
the  spacial  frequency  optical  modulation.  It  Is, 
however,  suitable  for  home  and  Industrial  TV 
projection  applications.  It  also  has  the  practical 
problem  that  each  of  the  three  CRTs  are  affected  by 
the  earth's,  as  well  as  manmade,  magnetic  fields 
in  a  different  manner  causing  long-term  con¬ 
vergence  problems— particularly  on  a  nonstationary 
system  such  as  a  moving  gun  turret. 

PRACTICAL  SYSTEMS 

Figures  5  and  6*  ^.^fst^ate  two  of  several 
configurations  which  proved  useful  In  telescope/ 
periscope  simulation.  SRL  i^AL'/eH'vnred  several  of 
each  of  these  Implementations^ iNr.g  the  latter  half 
of  1980  and  first  half  of  1981, the  gunner's  and 
commander's  sights  In  tank  cofT'jct  of  fire  trainers 
(COFT).  Both  configurations  have  proved  to  be 
capable  of  providing  the  required  color  resolution. 

One  version  used  a  ]?’  mm  CRT  with  a  108  mm 
useful  diagonal.  Shown  l^&jt.Vire  '/,  It  demonstrated 
over  1400  TV  lines  p:;r  plf'^e  nelght  and  width 
resolution  using  an  SRL  designed  and  fabricated  beam 
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combining  optics.  The  second  system,  shown  In 
Figure  8,  Is  used  In  General  Electric's  successful 
COFT  prototype.  It  uses  a  modified  color  TV  camera 
prism  assembly  to  combine  the  images  of  the  three 
CRTs.  Even  though  the  CRTs  only  had  a  40  mm  useful 
screen  diagonal,  It  exhibited  1000  TV  lines  per 
picture  height  horizontal  resolution.  Both  of 
these  two  systems  had  a  circular  format  Image  area 
where  the  square  CIG  raster  was  overscanned  such 
that  the  full  useful  diagonal  area  of  the  round 
CRTs  was  used  as  shown  In  Figure  9.  An  engineer  Is 
shown  In  Figure  10  aligning  the  General  Electric 
tr 1  noscope  display. 

Since  the  full  color  Image  from  a  trl noscope 
Is  derived  by  optically  overlaying  three  separate 
Images,  It  becomes  obvious  that  means  must  be 
provided  to  converge  the  three  Images  so  that  they 
appear  as  one.  For  lower  resolution  systems,  such 
as  home  TV  applications,  the  only  corrections 
needed  are  Individual  size  and  position  of  each  of 
the  three  rasters.  Keystone  (trapezoidal} 
correction  Is  also  needed  for  multiple  exit  pupil 
systems  such  as  shown  in  Figures  1  and  3.  But  fur 
high  resolution  systems,  more  elaborate  matching 
circuitry  Is  required.  Not  only  must  this  matching 
circuitry  allow  for  accurate  registration,  It  must 
be  very  stable  with  both  time  and  environmental 
changes. 

Figure  11  shows  a  simplified  block  diagram  of 
one  of  the  COFT  trl noscopes.  From  a  block  diagram 
viewpoint,  the  other  COFT  trinoscope  was  the  same 
except  that  raster  rotation  was  performed  In  the 
digital  Image  generation  equipment  Instead  of  In 
the  trinoscope  electronics.  To  keep  relative  drift 
between  the  three  channels  to  the  minimum,  much  of 
the  circuitry  Is  kept  common,  thus  common  mode 
drift  Is  close  to  zero.  This  Includes  the  sweep 
generation  circuits,  the  main  deflection 
amplifiers,  and  the  high  voltage  power  supply. 

The  convergence  circuitry  Is  included  to 
correct  for  the  differences  between  the  electrical, 
optical,  and  mechanical  characteristics  of  the 
three  channels.  Therefore,  each  channel  must  be 
Independent  and,  as  such,  can  drift  Independently 
of  each  other.  Several  factors  help,  however,  to 
make  this  drift  manageable.  Without  this 
correction,  the  three  channels  would  probably  match 
to  within  one  percent  due  to  the  selection  of 
matched  magnetic  components  and  CRTs  and  have  no 
more  than  0.25  percent  drift  with  time  In  normal 
simulator  environments.  This  basic  stability  Is 
achieved  by  use  of  SRL  linear  current  feedback 
deflection  amplifiers  which  even  correct  for  change 
in  the  deflection  yoke  resistance  with  temperature. 
Due  to  a  patented  power-on-demand  feature,  these 
amplifiers  are  significantly  more  power  conserving 
than  conventional  linear  deflection  amplifiers  and 
are  particularly  suitable  for  TV  raster 
applications.  (2)  The  convergence  circuitry  then 
needs  only  to  correct  for  the  one  percent  residual 
error.  By  using  temperature  stabilized,  linear 
feedback  current  amplifiers  for  the  convergence 
amplifiers,  It  is  safe  to  assume  that  the 
convergence  circuits  also  only  drift  an  amount  In 
the  order  of  0.25  percent  of  full  output.  Drift, 
then,  reflected  to  the  CRT  Is  0.25  percent  times 
one  percent,  or  l/40th  of  a  pixel  element  In  a 
1000  x  1000  pixel  element  system.  Of  course,  there 
will  be  differential  heating  and  other  second  order 
effects  In  the  system  that  will  cause  some 
mlsconvergence  errors,  but  suffice  to  say, 
convergence  drift  can  be  made  to  be  Insignificant. 


It  Is  difficult  to  measure  mlsconvergence 
through  a  telescope/perl  scope  optics  system. 
However,  SRL  was  able  to  consistently  achieve  a  one 
pixel  element  center,  2  or  3  pixel  element  edge 
mlsconvergence  on  a  1400  «  1400  pixel  element 
system.  This  amount  of  mlsconvergence  is  barely 
visible  with  a  dot  test  pattern  but  is  seldom 
distracting  with  real  life  type  CIG  scenes  such  as 
used  In  simulation  and  training. 

To  Illustrate  the  visual  system  capability,  a 
series  of  photos  from  General  Electric's  highly 
successful  Ml  Conduct  of  Fire  Trainer  Is  presented. 
Figure  12  Illustrates  the  full  CIG  scene  (taken  from 
a  conventional  19-Inch  monitor),  which  would  be 
observed  through  the  commander's  biocular  periscope. 
Figure  13  shows  the  3X  scene  as  It  would  be  seen 
through  the  Gunner's  Primary  Sight  as  he  ranges  In 
on  the  target,  and  Figure  14  shows  the  10X  view  for 
the  same  sight  as  a  hit  is  scored.  Note  the  high 
quality  of  the  red  rectlcle  and  the  green  numerics 
of  the  laser  rangefinder  produced  by  the  visual 
system.  Figure  15  depicts  a  "white-hot"  thermal 
Image  produced  from  the  same  data  base  which  further 
Illustrates  the  flexibility  of  a  CIG  visual  system. 
The  basic  configuration  of  this  mobile  trainer  Is 
Illustrated  In  Figure  16,  which  also  highlights  the 
premium  placed  on  minimal  optical  path  length. 

GE's  Ml  COFT  system  consists  of  four  simulated 
high  power  sights  driven  by  two  trinoscopes  and  a 
two-channel  CIG.  Beamsplitters  divide  the  Images 
such  that  the  commander's  two  sights  are  driven  by 
one  trinoscope  while  the  gunner's  two  sights  are 
driven  by  the  other  trinoscope.  Clever  "In-use" 
sensors  tell  the  CIG  which  sight  Is  being  used  so 
that  It  will  provide  the  proper  scene. 

As  mentioned  previously,  the  commander  also  has 
a  rectangular  IX  periscope  which  Is  driven  by  a 
standard  19-Inch  monitor  and  collimating  optics. 
Figure  17  shows  the  Interior  of  the  crew 
compartment.  All  five  sights  are  clearly  visible. 

SUMMARY 

The  use  of  trl  noscopes  In  telescope  and 
periscope  simulation  has  proved  practical  In  tank 
simulators.  Trinoscopes  would  be  equally  effective 
for  submarine,  antiaircraft  gun,  rocket  launcher  and 
any  other  system  which  views  a  target  or  scene 
through  an  optical  eyepiece  and  where  optical  path 
length  Is  at  a  premium. 

Since  there  Is  nothing  In  the  direct  view  or 
projection  trinoscope  concept  that  limits  It  to 
raster  scan,  a  calligraphic  (stroke  writing)  or  a 
dual  calllgraphlc/raster  version  could  be  made  using 
existing  circuitry. 

Although  the  tri noscope  concept  Is  very  old. 
Its  uses  are  as  new  as  the  next  generation  simulator 
or  trainer. 
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Figure  1.  Trinoscope  with  Three  Lenses 
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Figure  2.  Trlnoscope  with  Three  Off-Set  Lenses 
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Figure  3.  Trinoscope  with  Two  Lenses 


Figure  4.  Trlnoscope  with  Crossed  Dichrolcs  Image  Combiner  and  Single  Lens 
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Figure  5.  Trinoscope  with  Mirrors  and  Dichrolc  Image  Combiners  Suitable 
for  Telescope/Periscope  Simulation 


CRT  (8) 


LENS 


OBSERVER 


Figure  6.  Trinoscope  with  Dlchroic  Prism  Image  Combiner  Suitable 
for  Telescope/Periscope  Simulation 


Figure  7.  Top  View  of  Beam  Combiner  Type  COFT  Electro-Optical  Assembly 
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Figure  8.  Top  View  Figure  of  Prism  Type  COFT  Electro-Optical  Assembly 
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Figure  9.  Circular  Field-of-View  Images  of  Trinoscope  Systems  Shown  in  Figures  7  and  8 


Figure  10.  An  Engineer  Aligns  GE's  Trinoscope  at  SRL.  The  three  CRTs  are  mounted 
on  a  base  plate  assembly  along  with  collimating  optics  (white  tube). 


Figure  12.  CIC  IX  Scene  Presented  to  the  MI  Commander's  Periscope  Sight.  Note  middle  of  scene  for 
subsequent  figures. 


Figure  13.  CIG  Scene  Taken  Through  the  Gunner's  Primary  Sight  (GPS)  at  3X.  Note  excellent  registration. 
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Figure  16.  The  General  Electric  Conduct  of  Fire  Trainer  (COFT) 
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Figure  17.  The  Interior  of  GE's  COFT  Crew  Compartment.  Note  the  two  gunner's  high  power  sights  (lower). 
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ABSTRACT 


The  AMTESS  concept  Is  considered  to  be  a  forerunner  of  a  new  direction  in  maintenance 
training.  It  will  have  the  capability  of  providing  Introductory  maintenance  training  at  the 
institutional  level,  as  well  as  proficiency  training  In  the  field  environment.  The  system 
will  be  a  modular  and  flexible  maintenance  concept  with  broad  applicability  to  perform  In 
various  areas.  The  basic  AMTESS  requirements  were  generated  as  a  result  of  an  extensive  front 
end  analysis  by  four  highly  experienced  contractors  working  independently.  The  Air  Defense 
and  Ordnance  Schools  were  selected  for  the  Initial  analysis  effort.  As  a  besult,  each  con¬ 
tractor  provided  a  basic  design  of  a  system  which  would  meet  all  of  the  AMTESS  requirements. 
This  effort  constituted  Phase  I  of  the  program.  The  second  phase  of  the  program  resulted 
In  the  delivery  of  an  AMTESS  by  each  of  the  two  most  qualified  contractors  from  Phase  I, 

An  extensive  evaluation  period  Is  planned  by  the  Ordnance  school,  and  the  Air  Defense  School 
to  ensure  the  training  effectiveness  of  AMTESS.  The  Inherent  modularity  of  the  AMTESS  and 
the  ability  of  the  Instructors  to  modify  POI's  will  provide  a  device  with  a  wide  range  of 
flexibility  and  adaptability  to  overall  maintenance  training. 


INTRODUCTION  AND  BACKGROUND 

A  typical  maintenance  training  cycle  now 
being  conducted  at  the  schools  Is  depicted  In 
Figure  1.  Looking  at  the  cycle  In  reverse  order, 
however,  presents  a  representation  of  the  evolu- 
atlonary  process  associated  with  maintenance 
training  In  the  Army. 

Prior  to  the  Introduction  of  simulators  and 
training  aids,  the  majority  of  maintenance  train¬ 
ing  was  conducted  on  operational  equipment.  It 
became  obvious  early  on  that  there  were  several 
drawbacks  to  using  operational  equipment  for 
maintenance  training.  Granted,  It  did  present  the 
most  realistic  situation;  however,  it  was  difficult 
to  provide  efficient  and  effective  training. 

In  general,  operational  equipment  training 
required  a  one-on-one  student  Instructor  relation¬ 
ship.  Demonstrations  could  not  be  conveniently 
conducted;  faults  could  not  be  Inserted  easily, 
particularly  In  mechanical  equipment;  and  student 
assessment  was  based  on  a  subject  evaluation  by 
the  instructor. 

The  use  of  cut-aways  as  a  training  aid  was 
seen  as  a  major  breakthrough  by  many.  The  hous¬ 
ings  on  engines  were  cut  away  to  expose  the 
Internal  components.  Tank  turrets  were  even  cut 
open  and  mounted  on  stands  to  allow  the  Interior  to 
be  more  accessible  to  th?  students.  These  tech¬ 
niques  allowed  the  Instructors  to  demonstrate 
maintenance  procedures  to  several  students  simul¬ 
taneously.  However,  many  of  the  same  problems 
exist  as  when  standard  operational  equipment  was 
used  for  maintenance  training.  Faults  were 


difficult  to  simulate;  student  performance  could 
not  be  objectively  measured;  the  instructor 
student  ratio  was  still  high. 

The  introduction  of  the  panelboard  trainer 
was  considered  to  be  a  quantum  jump  in  the  area  of 
maintenance  training.  The  panelboard  trainer  with 
Its  associated  computer  and  visual  display  system 
provided  an  excellent  tool  for  classroom  Instruc¬ 
tion  as  well  as  Individual,  self  paced  Instruction. 

At  this  point,  elements  of  all  three  stages 
of  evolution  of  maintenance  training  were  com¬ 
bined  to  form  the  present  maintenance  cycle.  The 
two  dimensional  panel  board  allowed  the  Instruc¬ 
tor  to  provide  basic  Instructions,  insert  mal¬ 
functions,  and  assess  student  performance.  The 
students  were  able  to  learn  the  techniques  asso¬ 
ciated  with  Initialization  procedures,  trouble¬ 
shooting,  and  fault  isolation. 

Once  the  panelboard  techniques  had  been 
mastered,  the  student  then  proceeded  to  the  second 
portion  of  the  training  cycle,  the  three  dimension¬ 
al  mockup.  The  primary  emphasis  on  the  mockup  was 
remove  and  replace  procedures. 

The  three  dimensional  mockup  Is  not  a  neces¬ 
sary  Ingredient  in  all  training  situations.  How¬ 
ever,  It  has  been  found  that  in  complex  systems, 
students  have  difficulty  transitioning  from  the 
panel  board  trainer  to  the  operational  equipment. 
Despite  the  fact  that  the  student  may  have  mas¬ 
tered  the  panelboard  techniques,  some  students 
have  considerable  difficulty  locating  specific 
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components  on  the  operational  equipment,  There¬ 
fore,  the  mockup  serves  as  an  aid  to  familiarize 
the  students  with  the  overall  system.  In  addi¬ 
tion,  the  mockup  serves  to  obviate  the  physical 
constraints,  the  degree  of  difficulty  In  removing 
and  replacing  components,  and  the  particular  tools 
and  skills  Involved. 

The  final  stage  of  training  Is  on  the  opera¬ 
tional  equipment.  Eventually  the  trainee  must 
confront  the  operational  equipment  for  which  he  Is 
to  be  responsible.  Generally,  the  final  stage  of 
training  Is  on  the  job  training;  however,  In  many 
Instances,  operational  equipment  Is  dedicated  to 
maintenance  training. 

The  present  maintenance  training  cycle 
represents  the  most  effective  training  available. 
However,  each  stage  of  the  cycle  contains  basic 
deficiencies.  The  panelboard  type  trainers  lack 
flexibility.  Any  program  changes  or  configuration 
changes  require  a  fairly  major  modification  to  the 
system.  Hardware  and  software  changes  must  be 
accomplished  by  the  contractor. 

The  system  mockup  has  many  limitations.  It 
normally  provides  only  a  remove/replace  capabil¬ 
ity.  There  is  no  Interaction  with  a  central 
processing  unit  to  assess  student  performance, 
faults  cannot  be  Inserted  conveniently  Into  the 
mockup,  and  the  cost-benefit  ratio  If  not  favor¬ 
able. 

The  primary  drawback  with  using  operational 
equipment  In  the  final  stage  of  maintenance 
training  Is  the  Army's  emphasis  on  reducing  the 
use  of  operational  equipment  for  training. 

To  alleviate  the  deficiencies  associated  with 
the  present  maintenance  training  cycle,  the  AMTESS 
program  was  Initiated  In  June  1978.  This  program 
was  established  as  a  two-phase  effort.  Phase  I 
consisted  of  four  separate  tasks  associated  with 
the  Air  Defense  School,  Fort  Bliss,  Texas,  and  the 
Ordnance  School,  Aberdeen,  Maryland.  Under  Task  I, 
all  the  skills  associated  with  the  maintenance  of 
selected  automotive  and  missile  components  were 
analyzed  for  commonality,  and  representative  tasks 
were  selected  for  use  in  AMTESS.  Task  2  was  an 
analysis  of  the  training  requirements  for  the 
tasks  selected.  The  object  of  Task  2  was  to  deter¬ 
mine  the  types  of  skills  with  which  the  trainees 
were  to  emerge.  The  third  task  was  to  analyze  the 
fidelity  requirements  associated  with  each  phase 
of  training.  Task  3  was  particularly  significant 
since,  in  the  past,  there  has  been  considerable 
resistance  on  the  part  of  the  Instructors  to  use 
devices  which  did  not  exactly  duplicate  the  opera¬ 
tional  equipment  In  question. 

Despite  the  fact  that  studies  have  been 
conducted  to  validate  the  theory  that  exact 
replication  of  operational  equipment  Is  not 
always  necessary  to  provide  effective  training? 
instructors  remain  skeptical. 

Task  4  of  the  program  was  to  be  the  culmi¬ 
nation  of  the  first  three  tasks.  Based  on  the  data 
relating  the  tasks  associated  with  particular 
areas  of  maintenance  training,  the  training 
requirements  for  those  tasks,  and  the  level  of 
fidelity  required,  each  contractor  was  to  provide 
a  Preliminary  Systems  Engineering  design  of  a 
system  incorporating  those  data. 


The  primary  objective  of  Phase  II  of  the 
AMTESS  program  was  to  procure  an  AMTESS  In  accor¬ 
dance  with  the  Phase  I  design,  and  to  test  the 
system  for  training  effectiveness.  The  Army 
Research  Institute  (ARI)  was  tasked  with  the 
responsibility  of  providing  a  plan  for  determin¬ 
ing  the  training  effectiveness  of  AMTESS,  and  to 
monitor  the  evaluation  process. 

In  addition  to  providing  hardware,  a  secon¬ 
dary  objective  of  Phase  II  was  to  provide  a 
technique  for  defining  a  training  system  to  meet 
a  particular  training  requirement.  The  standard 
technique  for  establishing  training  programs  has 
been  to  start  with  a  training  device,  and  try  to 
fit  the  training  requirements  Into  that  device. 
Little  thought  was  generally  given  to  types  of 
devices  needed,  or  whether  a  device  of  sorts  was 
needed  at  all . 

The  Phase  I  portion  of  AMTESS  required  that 
a  thorough  front  end  analysis  be  conducted  prior 
to  designing  a  training  device  or  training  sys¬ 
tem.  It  Is  those  techniques  developed  In  Phase  I 
for  conducting  a  front  analysis  that  will  be 
preserved,  expounded  on,  and  delivered  In  Phase  II 
as  part  of  the  system  specification. 

In  that  form.  It  Is  anticipated  that  the 
specification  will  serve  as  a  guide  for  deter¬ 
mining  training  requirements  for  future  mainte¬ 
nance  training  situations.  The  outcome  which 
should  evolve  will  be  a  training  program  which 
meets  specific  training  requirements,  rather  than 
an  attempt  to  fit  training  requirements  Into  a 
particular  trainer. 


WHAT  IS  AMTESS  AND  WHAT  IS  IT 
SUPPOSED  TO  DO? 

The  requirements  for  AMTESS  are  extremely 
comprehensive.  Since  the  system  Is  envisioned  as 
the  forerunner  of  a  new  direction  in  maintenance 
training,  the  requirements  necessarily  Include 
almost  every  possible  maintenance  situation.  The 
following  are  some  of  the  required  features: 

o  Train  analytically  derived  requirements. 

o  Support  both  Institutional  and  unit 
training. 

o  Combine  heads-on  and  hands-on  training. 

o  Be  self  paced  and  adaptive. 

o  provide  for  automated  hands-on  adminis¬ 
tration  of  Skill  Qualification  Tests 
(SQT's). 

o  Exhibit  lowest  cost  of  ownership  for 
required  level  of  training  effective¬ 
ness. 

o  Apply  to  a  broad  range  of  Army  needs. 

o  Capitalize  on  advanced  technology 
developments . 

The  hardware  and  software  required  to  accomplish 
those  tasks  must  be: 

o  Modular  in  configuration  to  permit  ease 
of  component  Interchange  and  custom  con¬ 
figuration  for  the  particular  application. 

o  Closed-loop  In  design  to  provide  appro¬ 
priate  responses  to  student  inputs  and 
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errors,  to  provide  Instructional  cues, 
and  to  record  and  display  student  perform¬ 
ance. 

o  Generic  In  construction  to  assure  multi¬ 
ple-vendor  produclblllty,  low  cost,  and 
type  classification. 

o  Modifiable  by  Army  personnel  to  a1 low 
easy  updating  to  meet  changes  In  the 
equipment  or  Instruction. 

o  Adaptable  to  a  variety  of  Instructional 
uses  and  operating  environments. 

The  two  companies  (Grumman  and  Sevelle) 
awarded  contracts  under  Phase  II  were  selected 
primarily  on  the  basis  of  their  preliminary  de¬ 
sign  from  Phase  I.  Although  both  contractors 
were  required  to  meet  the  requirements  specified 
above,  no  restriction  was  placed  on  the  tech¬ 
niques  to  be  used  to  meet  those  requirements.  In 
fact,  divergency  In  technology  was  encouraged, 

The  primary  factor  In  determining  the  accept¬ 
ability  of  an  AMTESS  was  the  ability  of  the 
system  to  provide  effective  training.  In  evalua¬ 
ting  the  training  effectiveness,  If  It  was  found 
that  both  systems  were  equally  training  effective, 
cost  would  normally  then  be  the  determining  factor 
in  selecting  a  system.  Obviously,  if  one  system 
is  complex,  difficult  to  maintain,  and  Is  costly, 
and  the  other  system  Is  less  complex  and  lower 
cost,  but  has  basically  the  same  training  effec¬ 
tiveness,  It  would  be  difficult  to  justify  procur¬ 
ing  the  more  costly  system. 

Training  effectiveness  evaluations  will  be 
completed  In  May  1981.  A  decision  concerning 
system  selection  will  be  made  at  that  time. 

AMTESS  was  originally  envisioned  as  a  module 
modular  system  with  a  central  core  unit  as  shown 
in  Figure  2.  Although  some  of  the  earlier  tech¬ 
niques,  such  as  panelboard  trainers,  30  mockups, 
etc.,  could  be  Included  in  the  system  as  add-on 
modules,  each  module  would  Interface  with  the 
central  core  module.  The  two  dimensional  modules, 

3D  modules,  and  simulated  test  equipment  would 
all  be  active  elements;  l.e.,  would  respond  to 
the  trainees  Inputs,  and  provide  a  means  of 
evaluating  student  performance. 

The  primary  purpose  of  the  modular  concept 
Is  to  allow  a  high  degree  of  flexibility.  With 
the  control  unit  as  the  generic  portion  of  the 
system,  AMTESS  can  be  expanded  on  to  meet  the 
training  requirements  for  any  system  simply  by 
adding  the  system  peculiar  modules. 

As  the  system  designs  evolved  from  Phase 
II,  It  became  apparent  that  three  of  the  four 
designs  for  the  central  core  unit  were  basically 
the  same.  The  visual  display  technique  was  the 
main  difference  In  the  fourth  unit.  While  two 
units  employed  video  disc,  one  used  video  tape, 
the  fourth  unit  Included  a  random  access  slide 
projection  system. 

Of  prime  Importance  In  the  system  design  Is 
the  capability  to  modify  the  programs  of  In¬ 
struction  by  Instructor  personnel.  As  was 
pointed  out  earlier,  a  significant  deficiency  In 
some  of  the  present  systems  was  needed  for  the 
contractor  to  make  revisions  to  any  programs  of 
Instruction,  Even  minor  changes  generally  re¬ 
quired  that  the  panel  board  displays,  as  well  as 


the  software,  be  modified  at  the  contractor's 
plant.  With  the  recent  require"--^  ‘.hat  training 
devices,  particularly  malntenai,*  training  de¬ 
vices,  be  available  at  the  same  time  that  the 
operational  equipment  Is  fielded,  It  Is  Impera¬ 
tive  that  changes  to  the  operational  equipment  can 
be  Incorporated  Into  the  training  devices  as  they 
occur.  Mujor  configuration  changes,  or  drastic 
changes  to  the  program  of  Instruction  will  re¬ 
quire  contractor  modification.  However,  the 
Instructors  must  have  a  certain  degree  of  author¬ 
ing  capability  to  accommodate  minor  configuration 
and  program  changes. 

The  AMTESS  which  is  now  evolving  Is  typically 
shown  In  Figure  3.  The  central  core  unit  for 
this  particular  configuration  consists  of  a  CRT 
monitor  which  will  display  alphanumeric  charac¬ 
ters,  still  drawings  and  photographs,  and  motion 
pictures.  Visual  Images  are  generated  from  a 
videodisc  system  and  the  computer. 

The  CRT  will  also  include  a  touch  panel  capa¬ 
bility.  The  present  design  Includes  a  CRT/key¬ 
board  arrangement.  However,  this  function  may  be 
Incorporated  Into  the  touch  panel  with  a  projected 
keyboard  on  the  CRT  monitor.  The  touch  panel 
keyboard  will  actually  provide  greater  flexibility 
than  the  separate  CRT/keyboard  arrangement. 

The  computer  provides  the  Instructional 
program,  interfaces  with  the  three-dimensional 
module,  and  evaluates  the  trainee's  performance. 
Much  of  the  Information  associated  with  perform¬ 
ance  can  be  printed  to  provide  a  hard  copy  record 
for  each  trainee. 

Storage  will  be  either  on  a  floppy  disc  or 
rigid  disc,  depending  on  the  required  capacity 
for  the  particular  program  and  the  personal 
preference  of  the  contractor. 

The  system  depicted  ,n  Figure  4  represents 
the  configuration  of  an  automatlve  trainer  used 
In  engine  maintenance  training.  The  simulated 
engine,  and  the  components  associated  with  the 
engine  interface  with  the  core  unit.  In  addition 
to  engine  components,  simulated  test  equipment 
such  as  Simplified  Test  Equipment,  Internal 
Combustion  Engine  (STEICE),  multimeters,  and 
other  types  of  equipment  can  be  used  to  produce 
the  same  readings  as  indicated  on  the  operational 
hardware. 

The  second  area  of  training  which  was  of 
concern  to  this  phase  of  AMTESS  was  the  Air 
Defense  School's  radar  transmitter.  To  teach 
this  phase  of  maintenance  requires  only  that 
simulated  equipment  for  the  radar  shown  In 
Figure  5  be  substitute  for  the  automotive  com¬ 
ponents.  In  addition,  the  video-disc  and  the 
computer  program  must  be  replaced  with  the 
appropriate  radar  program. 

In  other  less  complex  training  situations, 

It  may  be  possible  to  provide  adequate  training 
with  the  core  unit  through  the  Interactive  CRT. 

CONCLUSION 

AMTESS  will  not  solve  all  the  problems 
associated  with  the  vast  chore  of  providing  main¬ 
tenance  training.  However,  It  is  an  attempt  to 
present  an  organized  method  of  determining 
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training  needs  and  will  provide  a  method  of  train¬ 
ing  a  wide  variety  of  maintenance  tasks.  In  a 
skillful  and  effective  manner.  If  this  system 
receives  universal  acceptance  within  the  Army, 
and  eventually  In  the  other  branches  of  the 
military,  two  things  will  be  accomplished.  First, 
a  means  of  providing  efficient  and  effective  train 
Ing  to  maintenance  personnel  will  be  possible. 
Second,  It  will  make  It  possible  to  have  a  base 
from  which  to  build,  thereby  precluding  the  need 
to  essentially  start  from  scratch  each  time  there 
Is  a  requirement  for  a  maintenance  trainer.  If 
AMTESS  Is  successful  In  accomplishing  these  two 
ends,  the  program  will  be  considered  a  success. 


(*)  Reduced  Physical  Fidelity  Training  Device 
Concepts  for  Army  Maintenance  Training,  Sept.  1978 
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ABSTRACT 

A  research  plan  was  developed  to  evaluate  the  cost  and  training  effectiveness 
of  the  F-16  SAMTs.  Historically,  such  evaluations  have  been  conducted  by  comparing 
the  effectiveness  of  the  simulator  against  that  of  an  actual  equipment  trainer 
(AET) .  However,  readily  comparative  training  devices  ard  approaches  do  not  exist 
in  the  case  of  the  F-16.  To  evaluate  the  training  effectiveness  of  the  SAMTs,  a 
criterion  referenced  approach  was  selected.  Students  will  be  assessed  on  their 
ability  to  perfcrm  maintenance  tasks,  taught  using  the  SAMTs,  on  actual  F-16  air¬ 
craft.  End-of-course  measures  and  follow-up  retention  testing  will  be  conducted. 
Engine,  pneudraulic,  electrical,  and  flight  control  system  tasks  will  be  evaluated. 
Task  selection  criteria  include  difficulty,  criticality,  and  frequency  of  perfor¬ 
mance.  Specific  training  capabilities  of  the  SAMTs  to  be  assessed  include  two 
instructional  features:  the  malfunction  insertion  capability  and  automatic 
student  monitoring.  A  comparison  will  be  made  between  the  performance  of  students 
trained  with  the  malfunction  insertion  feature  operational  versus  without  this 
instructional  capability.  The  use  of  the  student  monitoring  capability  will  be 
assessed  through  interviews  with  the  course  instructors.  A  comparative  approach 
was  adopted  for  assessing  the  cost  effectiveness  of  the  SAMTs.  The  cost  of  the 
hypothetical  AET  delivery  system  with  the  same  set  of  learning  objectives  as  the 
courses  the  SAMTs  are  utilized  in  will  be  computed  and  compared  to  the  SAMT 
delivery  system.  Major  categories  in  the  cost  model  include  facilities,  instruc¬ 
tional  equipment,  instructional  materials,  personnel,  and  supplies. 
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A  PLAN  FOR  THE  EVALUATION  OF  THE  F-16  SIMULATED  AIRCRAFT  MAINTENANCE  TRAINERS  (SAMTs) 


INTRODUCTION 


Traditionally,  maintenance  skills  such  as 
calibration,  inspection,  troubleshooting,  and 
repair  have  been  acquired  using  Actual 
Equipment  Trainers  (AET).  However,  the  use  of 
actual  equipment  in  the  training  situation  has 
not  been  without  practical  problems.  These 
problems  are  well  documented  in  the  literature. 
Some  of  the  problems  include: 

.  Extremely  Low  Reliability.  Actual 

equipment  is  sensitive  and  delicate  and 
in  the  training  situation  the  actual 
equipment  can  be  subjected  to  student- 
induced  damage.  Low  reliability 
results  in  low  availability  for 
training  purposes. 

.  Low  Maintainability.  Traditionally, 
spare  parts  for  training  has  had  a  low 
priority.  This  decreases  the  avail¬ 
ability  of  the  equipment  for  hands-on 
practice. 

.  Limitations.  Frequently  the  components 
or  unite  to  be  tested  cannot  be 
"failed"  in  the  ways  necessary  to 
provide  complete  and  meaningful 
troubleshooting  practice.  In  addition, 
emergency  conditions  often  cannot  be 
fully  practiced. 

These  problems,  coupled  with  the  generally 
high  initial  cost  of  AET,  encouraged  the  search 
for  a  viable  alternative. 

Growing  consideration  has  been  given  to  the 
concept  of  simulation.  Maintenance  training 
simulators  are  expected  to: 

.  Reduce  Cost.  The  initial  cost  and  the 
operational  and  maintenance  co3ts  of 
simulators  are  expected  to  be  lower. 
Furthermore,  a  lower  downtime  rate  means 
increased  availability  for  training. 

.  Improve  Training.  Simulators  should 
provide  improved  training  not  only 
through  higher  availability  rates,  but 
through  such  built-in  capabilities  as: 

-  Automatic  student  monitoring. 

-  Increased  availability  of  more 
varied  student  exercises  (more 
malfunction  identification  and/or 
correction  and  emergency  situation 
problems) , 

-  Immediate  feedback  to  reinforce 
correct  responses. 


-  Programed  remedial  instruction  with 
in-depth  explanation  of  course 
content . 

-  A  trainer  with  built-in  capabilities 
can  function  with  less  instructor 
dependency;  i.e.,  students  can  engage 
in  practice  without  the  need  for  an 
instructor,  thereby  decreasing 
instructor  demand. 

Wilh  these  expectations  in  mind,  the  USAF 
acquired  a  set  of  F-16  Simulated  Aircraft  Main¬ 
tenance  Trainers  (SAMTs).  The  SAMTs  are  the 
first  in  an  anticipated  series  of  maintenance 
trainers  for  major  weapons  systems.  As  such, 
the  SAMTs  represent  an  opportunity  to  determine 
if  the  expectations  of  reduced  cost  and  improved 
training  capability  have  been  realised.  Plans 
for  assessing  the  training  and  cost  effective¬ 
ness  of  these  simulators  are  described  in  the 
following  sections. 


APPROACH 


Training  effectiveness  and  cost  effective¬ 
ness  evaluations  of  simulators  are  not  a  new 
idea.  Historically  such  evaluations  are 
conducted  by  comparing  the  cost  and  training 
effectiveness  of  the  simulator  against  the  cost 
and  training  effectiveness  of  actual  equipment 
trainers  (AETs);  i.e.,  in  the  past,  readily 
comparative  devices  and  training  approaches  have 
existed.  This  is  not  the  case  with  the  selected 
F-16  SAMTs;  no  F-16  AETs  exist  nor  are  there  any 
plans  to  acquire  AETs.  In  addition,  because  of 
the  newness  of  the  curriculum,  there  is  no  base¬ 
line  data  on  past  student  performance  and 
proficiency.  Thus,  the  possibility  of  comparing 
the  performance  of  students  trained  on  the 
selected  SAMTs  with  the  performance  of  students 
who  have  been  trained  using  an  alternative 
training  program  or  approach  does  not  exist. 

The  approach  used  to  evaluate  the  SAMTs, 
as  well  as  the  results  of  the  evaluation,  will 
influence  the  acquisition,  design,  utilization, 
and  evaluation  of  future  maintenance  training 
simulators.  The  problem  is  to  develop  a  practi¬ 
cal  cost  and  training  effectiveness  evaluation 
plan  which  generates  accurate  and  useful  data, 
given  that  there  is  no  easily  identifiable 
comparative  training  approach.  Separate  plans 
were  devised  to  evaluate  training  and  cost 
effectiveness. 

Cost  Effectiveness 

The  cost  model  developed  for  the  current 
evaluation  is  based  on  a  review  of  the  most 
relevant  information  from  Air  Force  directives 


Cost  Categories  and  Features 


and  regulations  and  recent  economic  analysis 
publ Leal  ions . 

Basically,  a  comparative  approach  was 
adopted  for  assessing  the  cost  effectiveness  of 
the  SAMTs.  The  cost  model  will  be  used  to 
compute  simulator  costs  and  to  estimate  the 
costs  whicli  would  I  ave  been  incurred  with  the 
purchase  of  (hypotnet  i'cal )  actual  F-16  hardware 
equipment  trainers.  These  costs  will  then  be 
compared,  The  concert;  in  the  cost  effective¬ 
ness  evaluation  is  with,  comparing  the  costs  of 
the  two  types  of  instructional  delivery 
systems.  The  media  used  in  each  type  of 
delivery  system  is  only  part  of  the  cost  of  the 
system.  That  is,  the  SAMT  devices  and  hypo¬ 
thetical  AETs  represent  only  a  portion  of  the 
cost  required  to  conduct  the  courses  which  use 
the  two  types  of  media.  For  example,  the 
selection  of  media  may  influence  personnel 
costs,  facility  costs,  etc.  Thus  in  conducting 
the  cost  effectiveness  evaluation,  considera¬ 
tion  must  be  given  to  the  total  delivery  system 
and  not  just  to  the  type  of  instructional 
equipment  used. 

Assumptions 

There  will  be  a  hypothetical  AET  course 
for  each  SAMT  delivery  system.  These 
corresponding  AET  delivery  systems  will  be 
designed  to  have  have  the  same  set  of  learning 
objectives  as  the  corresponding  SAMT  delivery 
systems.  It  will  be  assumed  that  both  types  of 
delivery  systems  are  equally  training  effective, 
although  it  is  possible  for  the  courses  to  be 
different  with  respect  to  the  student  flow, 
number  of  instructors  required,  facility  sixe, 
instructional  material,  and  supplies.  This 
approach  has  several  advantages: 

Any  other  assumptions  of  AET  training 
effectiveness  would  have  to  be  based  on 
subjective  judgment. 

.  This  approach  allows  the  two  competing 
delivery  systems  to  be  compared  purely 
on  costs. 

.  The  assumption  of  equal  training  effec¬ 
tiveness  makes  only  two  results 
possible: 

Equal  benefits  and  equal  costs. 

Equal  benefits  and  unequal  costs 

.  There  is  some  evidence  in  the  litera¬ 
ture  that  support  the  assumption  of 
equal  training  effectiveness  between 
the  two  delivery  systems. 

Another  assumption  is  that  the  economic 
life  of  the  SAMTs  is  identical  to  their 
physical  life  (15  years).  Additionally,  the 
economic  life  of  the  AETs  will  also  be  assumed 
to  be  15  years. 

Finally,  it  is  assumed  that  the  SAMTs  and 
AETs  are  fixed  equipment,  and  that  the  variable 
equipment  required  by  each  delivery  system  are 
identical . 


The  cost  model  developed  for  use  in  the 
F-16  SAMT  evaluation  effort  has  the  following 
major  cost  categories: 

.  Facility  costs. 

.  Instructional  equipment  costs. 

.  Instructional  materials  costs. 

.  Personnel  costa  (cost  of  salaries  and 
benefits  to  instructors  and  student;). 
Supply  costs. 

,  Miscellaneous  costs. 

Each  of  the  major  cost  factors  or  categories  is 
subdivided  into  associated  costs,  These 
subfactors  define  the  costs  that  compose  the 
major  cost  factor.  Subfactors  may  reflect  cost 
areas  such  as  acquisition,  operation,  and 
maintenance.  These  subfactors  are  further 
divided  corresponding  to  the  cost  area  (for 
example,  equipment  acquisition  costs  can  be 
further  separated  into  procurement,  shipping, 
and  installation  costs). 

Although  the  cost  model  was  compiled 
specifically  for  this  project,  it  is  general  in 
nature  and  can  be  applied  to  the  costing  of 
other  types  of  delivery  systems.  The  cost 
model  has  the  following  features: 

.  It  considers  research  and  development 
costs  as  sunk  costs. 

.  It  uses  the  present  value  cost  concept 
to  account  for  differential  cash  flow 
pat  terns  between  the  competing  delivery 
system.  The  present  value  cost  method 
is  used  lo  account  for  possible 
differences  in  cash  flow  for  each  year 
of  the  comparison  period  for  each 
delivery  system  being  compared. 

.  It  separates  implementation  costs  from 
the  costs  incurred  in  subsequent 
years . 

.  It  divides  the  instructional  equipment 
cost  category  into  types  of  equipment : 
Fixed  Equipment  and  Variable  Equip¬ 
ment  . 

,  It  is  comprehensive  in  its  list  of 
subfactor  costs;  i.e.,  it  includes  all 
relevant  cost  categories  and 
subcategories  for  training  systems. 

.  It  considers  the  costs  associated  with 
updating: 

-  The  instructional  equipment  (SAMTs 
and  AET). 

-  The  instructional  features 
so  ft ware/ courseware. 

-  All  types  of  instructional 
materials , 
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.  It  allow*  for  removing  from  the  pretent 
value  cost,  the  remaining  value  of  the 
following  at  the  end  of  the  compariaon 
period : 

Facilities. 

-  Instructional  Materials. 

Instructional  Equipment. 

.  It  separates  the  utility  costs  required 
to  operate  the  facility  from  the 
utility  costs  required  to  operate  the 
fixed  equipment  (simulator  and  AET) . 

Comparing  Costs.  The  present  value  cost 
less  the  remaining  value  of  instruction 
materials,  equipment  and  facilities  will  be 
compared.  In  addition  to  comparing  the  total 
costs  of  each  delivery  system,  the  following 
analyses  will  be  made: 

.  A  comparison  of  the  implementation 
costs  of  both  types  of  delivery 
systems.  It  is  expected  that  one  type 
of  delivery  system  will  require  more 
front-end  money  than  the  other. 

The  costs  of  the  SAMT  delivery  systems 
will  be  compared.  It  will  be  of 
interest  to  note  which  SAMT  configura¬ 
tions  have  the  highest  and  lowest 
costs . 

,  The  major  contributing  costs  within 
each  delivery  system  will  be  exemined. 
Are  these  major  contributing  costs  the 
same  as  those  for  the  SAMT  delivery 
system? 

For  each  corresponding  delivery  system, 
accumulative  costs  per  student  will  be 
graphed.  If  the  two  types  of  corres¬ 
ponding  delivery  systems  are  different 
(by  10  percent),  then  a  break-even 
point  in  student  flow  will  be  calcu¬ 
lated.  This  will  al low  managers  to 
"see"  how  many  students  would  be 
trained  by  the  lower  cost  delivery 
system  for  the  same  cost  as  the  highest 
cost  delivery  system. 

.  For  each  corresponding  delivery  system, 
accumulative  costs  per  operating  hour 
will  be  graphed.  If  the  two  types  of 
corresponding  delivery  systems  are 
different  (by  10  percent),  chen  a 
break-even  point  in  operating  hours 
will  be  calculated. 

.  Within  the  SAMT  delivery  system,  the 
costs  of  the  instructional  features 
will  be  compared. 

It  is  anticipated  that  the  above  compari¬ 
sons  will  assist  the  Air  Force  in  determining 
the  co8t/benefit  derived  from  both  AET  and  SAMT 
type  delivery  systems.  Furthermore,  it  will 
provide  some  insight  into  the  cost  of  the 
instructional  features  under  study. 


Training  E t f ect iveaess 

A  criterion  referenced  evaluation  will  be 
conducted.  Basically,  this  technique  consists 
of  evsluating  students  on  their  ability  to 
perform  the  tasks  (objectives)  presented  in  the 
course,  after  they  hsve  been  exposed  to  the 
training.  If,  after  training,  the  trainees  can 
perform  what  the  course  was  designed  to  impart, 
then  the  training  can  be  judged  successful  or 
effective.  This  training  effectiveness  study  is 
designed  to  determine  if  the  graduates  of  F-16 
maintenance  training  programs  (which  employ 
SAMTs)  can  attain  criterion  on  the  learning 
objectives  specified  in  the  course  documents. 

Rationale  for  Selection  of  Approach.  There 
are  three  strong  reasons  for  selecting  a  criter¬ 
ion  referenced  assessment  approach.  First,  the 
absence  of  F-16  hardware  trainers,  with  the 
exception  of  the  F-100  engine  AET  utilized  in 
F-15  maintenance  training  (the  F-15  and  F-16 
engines  are  highly  similar),  prohibits  the 
adoption  of  a  comparative  approach  in  which  the 
effect ivenss  of  a  simulator  can  be  gauged 
against  the  effectiveness  of  an  AET. 

Although  no  AET  is  available,  the  F-16 
maintenance  courses  were  designed  to  be  taught 
with  either  the  SAMTs  or  on  actual  aircraft. 

This  would  provide  a  reasonable  basis  for  a 
comparative  study.  The  effectiveness  of  the 
SAMTs  and  the  effectiveness  of  the  actual 
aircraft  could  be  assessed  and  these  alternative 
training  approaches  compared  with  a  high  degree 
of  confidence.  In  fact,  the  design  of  such  a 
comparative  study  would  maximize  experimental 
control  (i.e.,  many  of  the  sources  of  extraneous 
variance  in  the  dependent  measures  could  be 
easily  controlled).  This  degree  of  control 
would  increase  the  likelihood  that  any  observed 
difference  in  student  performance  could  be 
attributed  solely  to  the  SAMTs,  since  both 
courses  would  be  identical  except  for  the  use  of 
the  aircraft.  However,  this  approach  raises  the 
following  issues: 

.  It  is,  perhaps,  unreasonable  to  expect 
the  Air  Force  to  teach  five  courses 
using  only  the  aircraft.  The  logistics 
problems  (for  example,  dedicating  air¬ 
craft  purely  for  training)  could  be 
overcome,  but  the  problems  of  guaran¬ 
teeing  that  the  same  instructor  teach 
both  groups,  that  the  courses  remain 
identical  to  the  SAMT  courses  (except 
for  the  use  of  the  aircraft),  and  other 
such  similar  problems  may  be  viewed  by 
the  FTD  as  an  unacceptable  and  costly 
burden . 

.  It  is  highly  unlikely  that  the  courses 
(using  just  the  aircraft)  have  been 
taught  before.  This  means  that  any 
comparative  study  would  be  "loaded"  in 
favor  of  the  SAMTs  since  part  of  the 
variance  in  the  comparative  dependent 
measures  would  be  the  "newness"  of  using 
the  aircraft  in  the  courses  (particu¬ 
larly  when  the  instructor  would  be 
accustomed  to  using  the  SAMTs). 
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Therefore,  a  second  major  reason  for 
selecting  a  criterion  referenced  approach  is 
that  it  requires  less  manipulation  of  the 
existing  training  environment,  as  opposed  to 
the  demands  of  conducting  a  comparative  study. 


question  is  moderated  in  the  following  way:  Can 
students  perform  as  expected,  and  how  much  of 
their  performance  is  due  to  the  SAMT3?  The 
answer  to  this  question  can  be  found  by  using 
the  following  basic  research  design: 


The  third  reason  for  selecting  the 
criterion  referenod  approach  ia  to  best  meet 
the  project  objective  for  development  of  a 
general  assessment  model.  While  the  Air  Force 
will  undoubtedly  continue  to  purchase  some  AST, 
it  is  unlikely  that  there  will  be  much 
opportunity  for  comparative  studies  in  the 
future. 

Because  of  these  reasons,  a  criterion 
referenced  approach  was  selected.  This  ia  not 
to  imply,  however,  that  a  criterion  referenced 
approach  is  easier  to  conduct  or  without 
research  design  problems. 

Disadvantages  of  Criterion  Referenced 
Approach i  In  a  criterion  referenced  approach, 
isolat ing  the  contribution  due  to  the  SAMTs  is 
much  more  difficult  than  in  a  comparative 
approach.  The  SAMTs  are  not  used  in  isolation. 
The  training  objectives  are  achieved  in  a 
variety  of  ways,  and  it  is  important  that  the 
evaluation  methodology  accurately  separate  the 
contribution  of  the  SAMTs  in  achieving  the 
objectives  from  the  contribution  made  by  the 
other  course  media  and  methods  (videotapes, 
slides,  aircraft,  and  OJT  experiences).  A 
comparative  study  would  be  much  easier  to 
conduct  from  this  perspective  than  a  criterion 
referenced  assessment. 

A  second  difficulty  is  that  the  criterion 
referenced  approach  requires  clearly  stated 
course  objectives.  The  learning  objectives 
appearing  in  the  F-16  course  control  documents 
(CCDs)  are  terminal  objectives.  For  the  most 
part  they  are  skill  or  performance-oriented; 
critical  enabling  objectives  (locating  or 
naming  parts)  are  not  identified.  Also,  some 
of  the  objectives  require  amplification  or 
clarification  by  instructors.  Finally,  some  of 
the  terminal  objectives  listed  in  the  CCDs  may 
not  match  what  is  expected  on  the  job.  The 
evaluation  methodology  must  validate  the  stated 
course  objectives  and  proficiency  levels,  and 
verify  their  consistency  with  actual  job 
requirements  and  expectations  of  field  super¬ 
visors.  Validation  is  needed  since  it  is 
possible  that  the  tasks  are  not  longer 
performed  in  the  same  manner. 

One  final  drawback  of  the  criterion 
referenced  approach  is  that  it  does  not  result 
in  direct  comparative  information.  That  is,  it 
will  not  be  possible  to  state  that  the  SAMT 
training  program  is  better  or  worse  than  some 
other  training  approach,  only  whether  or  not 
the  SAMTs  adequately  train  personnel  to  meet 
the  stated  course  objectives. 

Explanation  of  Design.  In  a  criterion 
referenced  approach,  the  main  concern  is 
whether  or  not  the  students  can  perform  as 
expected.  (Did  the  students  reach  the  stated 
course  objectives?)  In  the  present  case  this 


X  [Oyj ,  0wj * • • * i Own 1  0 i 

where  X  denotes  the  course,  within- 
course  measures  (to  isolate  the  effects  of  other 
media),  and  Oi  denotes  the  dependent  perform- 
ance  measure  taken  iosediately  following  the 
course.  (Since  the  use  of  various  medis  within 
the  course  is  fairly  well  blocked — generally 
only  one  type  of  media  is  used  within  an 
instructional  segment  and  the  courses  typically 
call  for  a  written  within-course  exam  at  the  end 
of  each  block — it  should  be  possible  to  segre¬ 
gate  the  effects  of  these  other  media  through 
the  use  of  within-course  measures,  administered 
at  the  end  of  each  instructional  block  as 
needed . ) 

In  order  to  assess  how  well  students  retain 
their  training,  a  follow-up  measure  is  taken 
several  weeks  following  the  end  of  the  course. 
The  design  then  becomes: 

X  [0wi,  0w2““>°wn]  °1  °2 

Some  degree  of  comparison  is  possible  in 
the  research  study.  The  evaluation  of  the 
malfunction  insertion  capability  will  involve 
having  the  instructors  teach  some  classes  with 
the  instructional  feature  operational  and  other 
classes  with  the  capability  turned  off.  The 
performance  of  these  two  groups  of  students  will 
then  be  compared.  The  two  mode  design  thereby 
assesses  the  malfunction  insertion  capability  by 
measuring  training  effectiveness  of  the  SAMT 
(student  performance)  with  the  feature  opera¬ 
tional,  and  comparing  this  data  with  training 
effectiveness  of  the  SAMTs  when  the  feature  is 
turned  off. 

The  automatic  student  monitoring  feature 
will  be  assessed  in  a  different  manner. 
Instructors  will  be  interviewed  and  asked  to 
describe  how  they  used  the  monitoring  feature. 
This  information  can  then  be  used  to  structure 
or  reconnend  more  in-depth  study. 


SAMTs .  Not  all  of  the  F-16  SAMTs  are 
targeted  For  evaluation.  Trainers  for  the 
pneudraulic,  electrical,  flight  control,  and 
engine  systems  were  selected.  These  SAMTs  were 
chosen  because  they  represent  the  range  of 
maintenance  task  complexity  and  difficulty. 
Also,  the  selected  SAMTs  vary  in  configuration. 


Each  SAMT  consists  of  at  least  one 
simulator  panel  set  (SPS)  and  a  master  simulator 
control  console  (MSCC).  The  engine  operation 
procedures  SAMT  is  configured  as  a  simulated 
cockpit . 
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Tho  other  8PSa  art*  vortical  flat  display 
panels  which  pictorially  illustrate  the  location 
and  relationships  of  the  components  of  each  of 
the  major  aircraft  systems.  Both  cockpit  and 
vertical  flat  panel  trainers  are  included  in  the 
study. 

Also,  positioned  on  each  of  the  panels  are 
simulated  test  equipment  which  provide  the 
student  with  the  facilities  to  troubleshoot 
simulated  malfunctions  to  a  particular  location, 
and  to  simulate  removal  and  replacement  of 
defective  components.  Test  set  connection/ 
disconnection,  remove/ replace  operations  and 
visual  inspection  steps  consist  of  pushing 
annotated  buttons. 

The  MSCCs  are  composed  of  a  Honeywell 
series  6/36  mini-computer  with  keyboard/CRT 
(cathode  ray  tube)  input-output  mode,  random 
access  33mm  slide  projector  and  hardcopy  line 
printer.  The  MSOC  contains  the  hardware  and 
software  interfaces  which  operate  the  SPSs  and 
provides  the  instructional  software. 

The  simulators  are  designed  to  be  used  by 
•wo  students  at  a  time.  The  instructor  initial¬ 
ises  the  system  via  the  MSCC,  selects  the  lesson 
to  be  presented,  and  turns  the  simulator  over  to 
the  student.  The  student  then  attempts  to 
operate,  calibrate,  or  troubleshoot  the  system 
with  the  aid  of  the  applicable  technical  orders 
and  job  guides.  The  simulator  monitors  the 
progress  of  the  student,  administers  feedback 
via  CRT  or  slides,  and  records  time  and  error  as 
the  student  works  through  the  problem.  For  the 
most  part,  the  simulator  responds  to  the 
student's  inputs  as  would  the  actual  equipment. 
Obviously,  Lh'-re  are  limits  to  the  simulator 
responses,  and  these  are  generally  constrained 
by  tho  SAMT  hardware. 

Trainees .  Maintenance  trainees  come  from 
a  variety  of  backgrounds.  For  the  purposes  of 
this  study,  only  3-levels  (trainees  just  coming 
out  of  technical  school)  ,  and  5-levels  are 
being  evaluated.  Foreign  students,  civilians, 
and  higher  level  military  personnel  (7-  and 
9-levels)  are  excluded. 

Tasks .  Training  effectiveness  will  be 
evaluated  primarily  through  assessing  student 
performance.  Performance  testing  will  consist 
of  observing  students  during  fault  isolation 
procedures,  ops  checks,  and  locate/ider.tify 
drills  on  the  aircraft  (in  some  cases,  the 
tasks  have  to  be  performed  on  the  SAMT  instead 
of  the  aircraft).  Task  protocols  were 
developed  from  T.O.s  and  translated  into 
checklists  to  be  used  as  a  basis  for  scoring 
students.  These  observational  checklists 
detail,  in  a  step-by-step  fashion,  the  actions 
the  trainee  should  execute. 

For  earh  course,  a  set  of  tasks  were 
selected  to  use  as  a  basis  for  evaluating 
student  performance.  Basically,  two  types  of 
tasks  were  chosen  for  each  course:  operational 
checkouts  (procedural  tasks)  and  fault 
isolation  tasks  (problem-solving  tasks  which 


involve  a  knowledge  of  system  logic  to  find  the 
cause  of  an  aircraft  malfunction!.  Selection 
criteria  included  the  following: 

.  Representative  of  system  maintenance 
skills  and  knowledge, 

.  Task  difficulty  and  length. 

.  Frequency  of  task  performance  on  flight 
line. 

.  Support  equipment,  materials,  and 
personnel  requirements. 

.  Feasibility  of  installing 

nonfunctioning  components  in  the 
aircraft  to  present  a  malfunction 
situat  ion. 

.  Potential  danger  to  personnel. 

.  Potential  damage  to  aircraft. 

Data  Collection.  Other  data  to  be 
collected  include: 

.  Pretraining  information  (student  record 
files  and  profile  questionnaire). 

.  Wi th in-course  skill/knowledge  tests 
(currently  administered  by  instructors 
at  the  end  of  instructional  blocks). 

.  Standard  end-of-coorse  multiple  choice 
exams , 

.  Instructor  attitudes  (questionnaire).* 

.  OJT  log  (trainee  task  experience 
record) . 

.  OJT  supervisor  evaluation  of  .student 
performance  (questionnaire). 

,  Student  attitudes  (end-of-course  and 
post-training  questionnaires). 

Method ■  In  order  to  understand  the 
sequence  of  data  collection  and  testing,  it  is 
necessary  to  review/describe  how  maintenance 
training  is  conducted.  First,  students  attend 
training  at  a  technical  school.  Following  this 
basic  systems  education,  trainees  are  assigned 
to  an  Air  Force  Base  and  receive  their  in-depth 
aircraft  system  training.  This  is  the  SAMT 
portion  of  their  education.  SAMT  course  length 
ranges  from  3  to  26  days.  After  successful 
completion  of  the  SAMT  course,  trainees  go  to 
on-the-job  training  (OJT) . 


*  The  Instructor  Attitude  Questionnaire  will  be 
the  vehicle  for  gathering  information  on  the 
automatic  student  monitoring  feature.  Speci¬ 
fically,  they  will  be  asked  to  rate  the 
utility  of  this  feature,  list  strengths  and 
weaknesses,  estimate  the  amount  of  use  of  the 
feature,  describe  how  and  when  they  used  the 
feature  in  the  classroom. 
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The  study  assesses  performance  (and 
student  attitudes)  at  the  end  of  the  SAMT 
course,  and  again  approximately  six  weeks 
later,  In  addition  to  the  student  performance 
and  attitude  data  (end-of-course  and  retention) 
measures,  within-course  testing  (standard 
exams)  are  given.  These  measures  should  help 
in  isolating  the  effects  of  the  SAMTs  from 
other  course  media.  (Indeed,  Resides  slides 
and  overheads,  some  courses  involve  hands-on 
aircraft  training  along  with  SAMT  exercises.) 
The  data  collection  timeline  is  presented  in 
Figure  1. 
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1.  Student  File  Data/Profile  Questionnaire 

2.  Within-Course  Exams  (written) 

3.  End-of-Course  Exams  (written) 

4.  Student  Attitude  Questionnaire  (1) 

5.  End-of-Course  Performance  Measure 

6 .  OJT  Log 

7.  Student  Attitude  Questionnaire  (2) 

8.  Retention  Performance  Measure 

9.  Supervisor  Rating 

10.  Instructor  Attitude  Questionnaire 

Figure  1 

Sequence  of  Training 
and  Experimental  Measures 

Evaluation  of  Training  Effectiveness.  Of 
primary  concern  in  the  current  study  is  the 
proportion  of  students  trained  using  the  SAMT 
who  pass/fail  the  stated  course  objectives 
which  are  targeted  for  evaluation  or  assess¬ 
ment,  More  specifically,  the  following 
research  hypotheses  will  be  statistically 
tested : 

.  At  least  70  percent  of  the  students 
will  reach  the  stated  proficiency  level 
on  the  objectives  targeted  for  evalua¬ 
tion  at  the  end  of  SAMT  training. 

.  At  least  70  percent  of  the  students 

will  reach  the  stated  proficiency  level 
on  the  objectives  targeted  for  evalua¬ 
tion  at  the  time  of  retention  testing. 

.  At  the  end  of  the  course,  it  is 

hypothesised  that  the  proportion  of 
students  passing  the  objectives  to  the 
malfunction  turned-off  mode  will  be 
significantly  lower  than  the  proportion 
of  students  exposed  to  the  malfunction 
turned-on  mode. 


.  Six  weeks  after  the  SAMT  training, 

Lhere  will  be  no  significant  difference 
between  the  proportion  of  students 
passing  the  staled  objectives  (at  the 
stated  proficiency  levels)  in  either 
of  the  three  experimental  modes 
(malfunction  ON  and  malfunction  OFF). 

This  hypothesis  contends  that  any 
differences  between  the  three  modes  at 
the  end  of  training  will  be  "washed 
out"  by  the  time  the  retention  measure 
is  taken;  i.e.,  the  instructional 
feature  effect  will  disappear. 

.  The  proportion  of  students  passing  the 
stated  objectives  will  be  significantly 
greater  after  OJT  than  immediately 
after  the  SAMT  course,  provided  that 
the  OJT  experience  supports  the  train¬ 
ing.  The  converse  is  hpothesiaed  for 
those  students  who  have  OJT  experience 
which  does  not  support  the  training. 

The  OJT  log  will  be  used  to  determine 
whether  or  not  the  OJT  experience 
supports  the  training.  The  following 
rule  of  thumb  will  be  used.  If  a 
student  performed  the  task  targeted  for 
evaluation  three  times  or  less  during 
the  OJT  experience,  then  the  OJT 
experience  will  be  classified  as  not 
supporting  the  training, 

.  The  level  of  confidence  reported  on  the 
student  attitude  questionnaire  will 
increase  between  the  end  of  training 
and  the  time  of  the  retention  measure, 
provided  that  the  OJT  experience  sup¬ 
ports  the  SAMT  training.  The  converse 
is  hypothesised  for  those  students 
where  the  OJT  experience  does  not 
support  the  training. 

.  Positive  attitude  toward  the  SAMTs  will 
increase  from  the  end  of  training  to 
the  time  of  the  retention  measure,  pro¬ 
vided  that  the  OJT  experience  supports 
the  training.  The  converse  is  hypothe¬ 
sised  in  those  cases  where  the  OJT  does 
not  support  the  training.  Attitudes 
will  be  determined  from  the  student 
attitude  questionnaire.  Positive 
attitude  will  be  determined  using  the 
following  rule  of  thumb.  A  perfect 
positive  attitudes  score  is  one  in  which 
the  student  indicated  the  maximum  value 
for  each  item  on  the  questionnaire.  A 
perfect  negative  attitude  score  is  one 
in  which  .ne  student  indicates  the  mini¬ 
mum  value  for  each  item  on  the  question¬ 
naire. 

In  addition  to  the  above  formal 
hypotheses,  the  following  descriptive  indices 
will  be  calculated  in  order  to  provide  a  more 
meaningful  basis  for  interpreting  the  data: 
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The  backgrounds  of  students  not  reaching 
criterion  at  the  end  of  training  will  be 
summarized.  This  summary  might  help  to 
determine  which  type  or  kind  of  students 
benefit  least  from  the  SAMT  type  of 
training  program. 

The  average  knowledge  test  score 
(in-class  measure)  of  those  students 
reaching  and  not  reaching  criterion 
will  be  calculated.  A  direct 
statistical  comparison  will  not  be 
performed,  since  such  information  adds 
little  to  the  problem  at  hand. 

However,  it  should  be  realized  that 
knowledge  measures  may  help  to 
interpret  the  performance  data. 

The  supervisor  ratings  of  those 
students  failing  both  the  end-of-course 
measure  and  the  retention  measure  will 
be  calculated.  These  indices  will  be 
reported  along  with  the  degr.e  Lo  which 
the  OJT  experience  supported  the 
training  in  order  to  put  the 
supervisory  ratings  in  the  proper 
perspective. 

The  correlation  between  the  supervisor 
ratings  and  both  the  end-of-course 
performance  iseasures  and  the  retention 
measures  will  be  calculated.  These 
correlations  will  assist  in  determining 
if  performance  measures  can  be  predicted 
from  supervisory  ratings. 

The  observational  checklist  of  those 
studenLs  who  do  not  reach  criterion  at 
the  end  of  the  course  and  at  the  time 
of  the  retention  measure  will  be 
examined.  The  checklists  will  be 
screened  for  the  type  of  errors  that  are 
made.  A  summary  of  the  errors  may 
assist  trainer  designers  in  future 
efforts;  i.e.,  if  possible,  an  attempt 
will  be  made  to  trace  the  errors  to 
design  problems,  such  as  level  of 
f idel ity . 


Finally,  the  study  will  provide  a  model 
for  future  evaluation  efforts.  A  handbook  will 
be  designed  for  the  Air  Force  to  assist 
individuals  in  designing  and  conducting 
training  and/or  cost  effectiveness  analyses  of 
maintenance  training  simulators. 

The  approach  and  results  of  the  F-16  SAMT 
assessment  will  influence  the  acquisition, 
design,  utilization,  and  evaluation  of  future 
maintenance  training  simulators. 


The  results  of  this  study  will  be  used  in 
a  number  of  rays.  Primarily,  the  results  will 
be  used  to  make  recommendations  regarding  the 
cost  and  training  effectiveness  of  simulators 
and  associated  instructional  features  for 
teaching  maintenance  tasks. 

The  /.ir  Force  SIMSPO  will  be  in  a  better 
position  vO  decide  which  features  to  incorporate 
on  future  trainers  based  on  the  results  of  this 
study.  The  cost  and  training  effectiveness  data 
will  provide  insights  on  how  to  restructure  the 
training  development  process,  as  well  as  the 
cost/benefit  tradeoffs  for  incrementing  the 
simulator's  capabilities  in  terms  of  instruc¬ 
tional  features. 
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ABSTRACT 

This  paper  describes  the  concept  development  and  feasibility  demonstration  of  a  man- 
safe  cueing  system  for  the  impact  of  indirect  fire  in  a  simulated  battlefield  environ¬ 
ment.  This  system  provides  a  safe,  cost-effective  method  for  including  mortar  and  artil¬ 
lery  into  the  battlefield  training/evaluation  scenario  for  both  mounted  and  dismounted 
players.  This  cueing  system  is  based  on  a  lightweight  projectile  having  a  very  low  impact 
energy  and  carrying  a  flash/bang/smoke  generator.  This  projectile  is  launched  from  a  low 
pressure  pneumatic  device  that  is  capable  of  projecting  the  cue  to  ranges  of  200  to  300 
meters  using  a  simple  constant-mass,  constant-drag  projectile.  A  manually  controlled 
launcher  was  constructed  to  demonstrate  the  feasibility  of  the  cueing  concept.  A  remote 
controlled,  multiple  shot  launching  device  (providing  coverage  of  a  one  kilometer  diameter 
circle)  is  well  within  current  technology.  The  soft-nose  projectile  is  designed  to  have  a 
terminal  energy  similar  to  that  experienced  during  the  impact  of  a  served  tennis  ball. 
The  flash,  acoustic  and  smoke  cues  are  tailored  for  player  safety. 


INTRODUCTION 

Overview 

Increased  military  awareness  of  the  impor¬ 
tance  of  finding  cost  effective  methods  of  simu¬ 
lating  battlefield  conditions  for  the  purpose  of 
testing  concepts  and  training  personnel  has  led 
the  U.S.  Army  to  investigate  simulation  techni¬ 
ques  for  indirect  fire  as  well  as  small  arms  and 
direct  fire  weapons.  Examples  of  the  latter  are 
the  MILES  (Multiple  Integrated  Laser  Engagement 
System)  and  the  IDFSS  (Infantry  Direct  Fire  Simu¬ 
lation  System)  programs.  Current  technology  for 
the  simulation  of  indirect  fire  uses  ground- 
placed  flash/bang/smoke  (FBS)  generators,  3moke 
bombs,  referee-designated  impact  areas,  and  other 
techniques  which  detract  from  the  realism  or 
imperil  the  participants. 

The  Engineering  Experiment  Station  (EES)  of 
the  Georgia  Institute  of  Technology  entered  into 
a  one  year-three  month  research  program  to  show 
the  feasibility  of  an  air  launched  Indirect  fire 
cue  (1).  The  overall  objective  of  this  research 
was  to  perform  an  exploratory  analysis  of  various 
methods  to  accurately  and  remotely  place  a  cue  of 
an  exploding  shell  in  a  predetermined  area.  This 
cue  must  automatically  and  simultaneously  deliver 
a  triple-signature  display  involving  a  brief  but 
Intense  flash  of  light  and  audible  impulse,  as 
well  as  the  deployment  of  a  gaaseous  or  particu¬ 
late  3moke.  All  facets  of  the  system  were  inves¬ 
tigated  with  regard  to  minimizing  their  potential 
for  causing  injury  to  the  operator,  players,  or 
observers.  Attention  was  also  given  to  the  en¬ 
vironmental  impact  resulting  from  the  presence 
and  use  of  the  system. 


Background 

The  objective  evaluation  of  military  tactics 
and  doctrine  requires  scientifically  controlled 
experiments  of  free  play  war  game  exercises  in  a 
simulated  battlefield  environment.  The  support 
for  such  field  experiments  requires  a  sophisti¬ 
cated  range  instrumentation  and  data  collection 
system.  It  also  requires  a  realistic  simulation 
of  the  weapon  systems  (in  a  manner  that  is  player 
safe)  to  preserve  realism  for  the  player  ele¬ 
ments.  The  U.S.  Army  Combat  Development  Experi¬ 
mentation  Command  (USACDEC)  at  Ft.  Hunter- 
Liggett,  California,  has  established  a  facility 
of  this  type  for  use  with  dismounted  players, 
vehicle  mounted  players,  and  player  and  weapon 
elements  that  are  airborne  on  helicopters,  and 
combat  aircraft. 

Direct  fire  weapons  have  been  effectively 
simulated  at  the  Ft.  Hunter-Liggett  test  range 
through  the  use  of  co-boresighted  laser  devices 
that  are  attached  to  the  weapon  barrels.  These 
laser  units  trigger  electronic  casualty  assess¬ 
ment  devices  (photocells)  located  on  other  player 
elements  at  distances  consistent  with  the  effec¬ 
tive  range  of  the  weapon  being  simulated.  The 
laser  simulation  system  is  triggered  for  small 
arms  fire  by  the  firing  of  a  blank  cartridge  in 
the  weapon.  With  larger  weapons  such  as  a  TOW 
launcher  or  a  tank  gun,  the  event  of  a  firing  may 
or  may  not  be  accompanied  by  a  simulation  of  the 
firing  signature  (an  FBS  signature  from  a  weapons 
effect  simulator  located  at  the  launch  site). 
These  weapon  simulation  systems  have  provided 
many  meaningful  measurements  in  predefined  tac¬ 
tical  situations  through  the  range  instrumenta¬ 
tion  system  and  data  collection  facility  at  Ft, 
Hunter-Liggett.  The  simulated  battlefield  sce¬ 
nario  addresses  many  of  the  weapon  systems  en¬ 
countered  in  actual  battlefield  situations,  but 
it  does  not  include  a  simulation  and  casualty 
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assessment  for  Indirect  fire  weapons  (howitzers, 
mortars,  etc.).  Indirect  fire  weapons  may  be 
used  for  Inflicting  casualties  on  opposing  forces 
or  for  troop  suppression.  Inclusion  of  this 
important  area  of  weapon  technology  is  highly 
desirable  to  develop  a  realistic  and  accurate 
simulation  of  a  battlefield  environment.  The 
relatively  high  trajectory  and  long  firing  range 
of  the  indirect  fire  weapon  class  does  not  allow 
the  laser  Illuminator  to  be  used  as  it  is  with  a 
direct  fire  system.  The  simulation  of  the  indi¬ 
rect  fire  weapon  may  be  limited  to  the  impact 
signature  without  affecting  the  realism  to  the 
players  since  the  gun  position  would  normally  be 
several  kilometers  away.  This  impact  simulation 
and  casualty  assessment  is  best  accompanied  by  a 
suitable  simulation  of  a  PBS  cue. 

The  impact  simulation  of  indirect  fire  wea¬ 
pons  is  an  extremely  difficult  problem  to  solve 
in  a  manner  that  is  oost  effective  and  relatively 
safe  to  the  player  elements  in  the  field.  As  a 
result,  the  term  "simulation"  is  not  an  accurate 
description  due  to  the  requirements  for  man 
safety.  A  true  simulation  (normally  Implying  a 
reproduction  or  copy  of  the  original)  is  not  de¬ 
sired  due  to  the  possibility  of  human  injury  re¬ 
sulting  from  this  level  of  explosion.  In  the 
simulated  battlefield,  a  more  appropriate  term 
for  creating  the  impact  signature  from  an  indi¬ 
rect  fire  weapon  is  a  "cue"  containing  the  three 
basic  FBS  characteristics.  Each  of  these  cues 
must  be  tailored  to  provide  the  required  player 
safety  for  the  battlefield  scenario  employed. 

Evaluation  Criteria 

Evaluation  criteria  were  applied  to  each 
feasibility  model  design  task  to  optimize  the 
final  recommended  cueing  system.  The  evaluation 
criteria  were  (not  necessarily  in  order  of  impor¬ 
tance  ) : 

1 .  Performance 

2.  Cost 

3.  Reliability 

4.  Safety 

5.  Operational  Life 

6.  Environmental  Impact 

7.  Ease  of  Manufacture. 

The  performance  of  the  deliverable  system  is 
of  course  a  function  of  each  integral  component, 
however,  the  ultimate  test  of  performance  was 
judged  on  the  basis  of  how  accurately  an  FBS  pro¬ 
jectile  could  be  projected  to  a  desired  location 
and  to  what  degree  the  FBS  projectile  acted  as  an 
artillery  or  rocket  fire  cue  to  players  engaged 
in  a  simulated  battlefield  situation.  Accuracy 
wa3  determined  by  a  circular  error  probable  (CEP) 
analysis  of  the  projectile  impact  locations  at 
the  maximum  contract-specified  range  (150  me¬ 
ters).  The  definition  of  CEP  used  in  conducting 
this  analysis  is  taken  from  (2).  The  effective¬ 
ness  of  the  cue  was  based  on  the  ability  of  the 
FBS  unit  to  generate  an  attention-gaining  audio/ 
visual  event.  The  performance  of  the  deliverable 
system  and  that  performance  which  is  technically 
possible  differ  due  to  the  constraints  placed 
upon  the  system  by  the  Surgeon  General's  safety 
standards  as  well  as  cost  limitations.  Compl¬ 
iance  with  established  safety  and  health  regula¬ 


tions  placed  ceilings  on  various  parameters 
(e.g.,  sound  level).  Other  items  were  econo¬ 
mically  regulated.  For  example,  the  projectile 
could  employ  sophisticated  microelectronic  cir¬ 
cuitry  to  regulate  flight  characteristics,  timing 
of  FBS  deployment,  etc.,  however,  the  transferral 
of  as  much  technology  as  possible  from  the  pro¬ 
jectile  to  the  launching  device  is  important 
because  the  launcher  is  a  one-time  fixed  cost 
while  expendable  projectiles  are  a  recurring  cost 
and  therefore  should  be  as  economical  as  pos¬ 
sible.  As  is  usually  the  case,  economy  implies 
simplicity,  so  constraints  were  placed  on  the 
projectile  design  complexity. 

Operational  life  and  reliability  are  related 
factors  which  also  direoted  the  course  of  the 
design.  For  example,  the  decision  to  have  an 
expendable  projectile  (long  shelf  life  but  short 
operational  life)  was  a  step  toward  Increased 
reliability.  Reliability  was  also  considered  in 
terms  of  launoher  design  since  future  Army  goals 
require  the  launcher  to  operate  unassisted. 

The  predominant  risks  to  environmental 
safety  as  a  result  of  using  this  system  stem  from 
fire  hazard  and  toxic  materials  pollution.  The 
launcher  design  is  completely  free  of  these 
risks,  however,  the  projectile  design  necessi¬ 
tates  the  use  of  nonbiodegradable  materials  and 
minute  quantities  of  volitile  substances.  Care 
was  taken  to  avoid  toxic  materials  in  the  con¬ 
struction  of  the  projectile.  Ingestion  by  man  or 
animal  of  any  part  of  the  projectile,  though 
unsuitable  for  consumption  because  of  the  phy¬ 
sical  shapes  involved,  would  not  result  in  poi¬ 
soning  (especially  in  the  quantities  present). 
The  chance  of  starting  a  fire  is  remote  through 
careful  gas  containment  design  and  is  further 
diminished  by  the  use  of  nonflaunable  and  self 
extinguishing  materials  (e.g.,  polycarbonate 
sheet  (Lexan),  polystyrene  (Styrofoam),  teflon, 
etc.). 

Ease  of  manufacture  was  essential  to  provide 
an  affordable  and  usable  cueing  system.  Economy, 
reliability,  and  serviceability  all  stem  from 
uncomplicated  design.  Not  only  the  composition 
and  shape  of  each  system  component,  but  the 
underlying  design  philosophy  had  to  be  considered 
before  recommending  a  design  for  mass  production. 

FLASH/BANG/SMOKE  UNIT 

The  flash/bang/snoke  (FBS)  unit  contained 
within  the  projectile  has  been  engineered  to  de¬ 
liver  a  triple-signature  display  consisting  of  an 
intense  flash  of  light,  an  acoustic  impulse,  and 
a  cold  particulate  smoke.  The  respective  signa¬ 
tures  have  been  limited  to  bounds  established  by 
the  Surgeon  General  where  applicable,  and  where 
no  guidelines  are  available,  safety  limits  have 
been  Justified  and  established  through  experimen¬ 
tation  and  consultation  with  medical  specia¬ 
lists.  Fire  safety  has  been  enhanced  through  the 
use  of  cold  ejected  particulate  smoke,  high  effi¬ 
ciency  light  generation  with  triple  light-trans- 
mitting/heat-blooking  barriers  to  minimize  heat 
leakage  to  the  environment,  and  encapsulated, 
self-extinguishing,  or  fire-proof  projectile  and 
FBS  unit  components. 
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The  purpose  of  the  FBS  unit  is  to  deliver  a 
detectable  cue  during  war  games  so  individual 
players  will  realize  that  they  are  under  indirect 
fire.  A  secondary  purpose  for  the  FBS  unit  is 
that  it  will  in  a  limited  sense  simulate  the  ex¬ 
plosion  of  artillery  round  upon  lnpact.  The  most 
desirable  situation  from  a  oue  standpoint  and  a 
simulation  standpoint  would  be  to  have  the  FBS 
unit  identically  simulate  an  actual  artillery 
impact  and  yet  be  man  safe.  This  is  a  contra¬ 
diction,  however,  and  in  a  war  game  scenario, 
man-safety  has  priority  over  simulated  realism. 

Numerous  restrictions  have  been  placed  on 
the  FBS  unit  in  an  effort  to  assure  man  safety. 
First,  the  flash  should  not  exceed  that  experi¬ 
enced  during  a  normal  photographic  flash.  The 
exposure  to  acoustic  impulse  noise  is  designated 
by  the  Surgeon  General  of  the  U.S.  Army  (TB  MED 
251)  not  to  exceed  140  dB  peak  impulse  at  the  ear 
without  the  use  of  ear  protection  to  be  ear- 
safe.  A  distance  of  six  inches  from  the  ear  has 
been  interpreted  to  specify  the  sound  level  "at 
the  ear."  This  interpretation  is  justified  (in 
the  absence  of  any  other  guidelines)  as  being  a 
reasonable  estimation  of  what  might  be  experi¬ 
enced  by  a  player  in  the  prone  position  having  a 
cue  impact  on  the  ground  beside  his  head.  Thus, 
a  peak  level  of  109  dB  at  18  feet,  for  instance, 
would  be  nominally  140  dB  at  six  inches.  The 
consultants  issuing  TB  MED  251  have  designated 
the  140  dB  peak  value  as  being  unlikely  to  pro¬ 
vide  hearing  problems  on  successive  applications. 

The  smoke  associated  with  the  FBS  unit,  par¬ 
ticularly  when  used  to  cue  a  forward  observer, 
must  be  visible  at  a  range  of  one  or  two  kilo¬ 
meters.  A  relatively  dense  cloud  approximately 
two  feet  in  diameter  by  four  foot  in  height  has 
been  shown  to  be  a  minimum  for  two  kilometer 
visibility  (3).  This  nominal  cloud  size  was  the 
goal  for  this  development  program. 

Basic  Configurations 

Combinations  of  flash,  bang  and  smoke  con¬ 
sidered  for  the  FBS  unit  included  an  integral 
flash,  bang  and  smoke;  separate  flash  from  bang 
and  smoke;  as  well  as  various  means  for  producing 
each  (e.g.,  electrical  flash,  pyrotechnic  bang 
and  chemical  smoke).  The  success  of  any  given 
technique  as  it  relates  to  an  FBS  cue  is  somewhat 
subjective. 

A  commercially  available  Magicube  camera 
flashbulb  was  adopted  as  an  igniter  and  flash 
generator.  It  can  be  actuated  by  a  slight  impact 
of  a  small  wire  on  the  stem  of  the  flashbulb  and 
eliminates  the  need  for  an  electrical  ignition 
system  or  an  impact  primer  system.  The  Magicube 
bulb  puts  out  significant  light  with  a  5500° 
Kelvin  color  temperature.  The  heat  associated 
with  the  flash  is  such  that  it  can  be  used  to 
ignite  pyrotechnics,  thereby  acting  as  a  primer 
for  the  bang  and  smoke  charges  of  the  FBS  unit. 

The  bang  of  the  FBS  indirect  fire  cue  must 
be  sharp  enough  to  be  heard,  but  not  exceed  on  an 
impulse  basis  140  dB  peak  at  the  ear  of  the 
player.  The  magnitude  of  this  sound  impulse  is 
directly  related  to  the  rate  of  release  of  the 
pyrotechnic  gases  as  well  as  the  quantity  of  the 


gases.  The  release  rate  is  a  function,  there¬ 
fore,  of  the  rati  of  escape  of  a  sabot  or  the 
rate  of  rupture  of  a  container. 

Smoke  is  defined  as  a  cloud  of  particulate 
material  having  particles  between  .01  microns  and 
100  microns  of  such  number  and  concentration  that 
the  contrast  between  the  population  of  these 
partioles  to  nearby  particulate  affects  visibi¬ 
lity,  light  reflection  and  scattering.  As  de¬ 
fined,  smoke  may  be  composed  of  soot  partioles, 
dust  particles  or  almost  any  small  particulate 
matter  suspended  in  the  airmass.  Consequently, 
in  this  program  the  two  methods  of  providing 
smoke  were  to  let  a  pyrotechnic  develop  the  smoke 
during  the  combustion  process,  such  as  a  phos¬ 
phorus-oxygen  type  smoke,  or  to  provide  a  dust 
material  contained  in  a  sabot  which  is  dispensed 
into  the  air,  by  burning  a  small  amount  of  pyro¬ 
technic  powder  behind  the  sabot. 

Tests  show  that  pyrotechnic  smoke  performs 
better  than  ejected-dust  smokes,  however,  the 
general  fire  hazard  involved  in  a  pyrotechnic 
smoke  and  the  usually  detrimental  by-products  of 
the  burning  substance  which  forma  the  smoke  made 
pyrotechnic  smokes  a  less  desirable  choice  than 
the  ejected  dust  for  the  FBS  unit.  In  addition, 
ejected-dust  smokes  of  the  type  of  sodium  bicar¬ 
bonate,  potassium  bicarbonate  and  calcium  car¬ 
bonate  are  water  soluable  and  what  little  dust  of 
these  types  that  actually  gets  into  lung  areas 
will  be  dissolved  by  lung  fluids  and  subsequently 
be  ejected  with  other  waste  material. 

Bang  and  Smoke  Ignition 

Analysis  indicated  that  the  best  solution  to 
ignition  of  the  bang  and  smoke  section  of  the  FBS 
unit  was  to  utilize  the  caloric  output  of  the 
Magicube  (TM,  Sylvania)  flash  generator.  The 
zirconium  oxide  which  is  the  product  of  combus¬ 
tion  within  the  Magicube  bulb  boils  at  5000° 
Kelvin.  The  boiling  process  tends  to  stabilize 
temperature  and  hence  limits  the  peak  temperature 
that  is  achieved.  The  radiating  temperature  is 
therefore  5000°,  and  the  bulb  can  transfer  only 
about  one  to  two  calories  of  energy  into  its 
surroudings.  Ignition  with  this  system  is 
achieved  by  coating  the  inner  faces  of  the  Magi¬ 
cube  bulbs  (four  per  flash  cube)  with  a  rubber- 
based  cement  which  hold3  an  ignition  compound  in 
close  proximity  to  the  surface  of  the  bulb  for 
most  realiable  ignition.  The  "powdered-bulb" 
method  of  bang  and  smoke  ignition  has  several 
advantages  over  others  tried.  In  particular, 
realiability  is  increased  because  any  one  of  the 
four  flashbulbs  in  the  Magicube  can  ignite  the 
ignition  compound  and  thereby  result  in  detona¬ 
tion  of  the  bang  and  smoke  portion  of  the  FBS 
unit.  Testing  of  Magicubes  has  yielded  a  small 
number  of  bulbs  which  were  defective  for  one 
reason  or  another,  therefore,  this  redundant 
reliability  factor  is  warranted.  A  further  ad¬ 
vantage  is  that  the  ignition  compound  burns 
within  the  plastic  housing  of  the  Magicube;  this 
tends  to  minimize  the  chance  of  hot  gases  es¬ 
caping  into  the  atmosphere  and  causing  a  fire. 

The  bang  and  smoke  section  of  the  FBS  unit 
is  housed  in  a  16  gauge  shotgun  shell  as  shown  in 
Figure  1.  This  portion  of  the  FBS  unit  is  herme- 
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Figure  1 .  Flash/Bang/Smoke  Unit 


tically  sealed  since  the  bang  and  smoke  charge 
primer  is  hygroscopic  and  the  atomized  oalcium 
carbonate  "cold  smoke"  must  be  moisture  free  for 
correct  dispersion  when  deployed.  As  shown  in 
the  figure,  the  bang  and  smoke  charge  is  placed 
through  an  opening  in  the  top  of  the  Magicube. 
housing.  Upon  projectile  impact,  the  Magicube 
bulbs  are  actuated,  and  this  in  turn  ignites  the 
Ignition  compound  painted  on  the  bulbs.  The 
flash  of  the  Magicube  bulbs  and  the  ignition 
compound  constitutes  the  flash  signature.  The 
hot  gasses  produced  by  the  burning  of  the  igni¬ 
tion  compound  penetrate  the  hermetic  seal  over 
the  primer  charge  or.  the  bang  and  smoke  unit. 

These  hot  gases  continue  into  the  black 
powder  shotgun  shell  primer  causing  it  to  flash 
through  the  flash  hole  in  the  primer  cup  of  the 
shotgun  shell  and  ignite  the  smokeless  bang 
charge  contained  on  the  other  side.  When  the 
smokeless  bang  charge  detonates,  it  creates  ra¬ 
pidly  expanding  gases  which  serve  1)  to  cause  a 
bang  and  2)  to  blow  the  calcium  carbonate  "cold 
smoke"  charge  out  the  back  of  the  shell  (and 
through  the  weak  polystyrene  tail  section  of  the 
projectile),  thereby  deploying  the  smoke  signa¬ 
ture.  Since  the  smoke  is  a  cold  smoke,  the  fire 
hazard  is  diminished.  The  rapidly  expanding 
gases  which  drive  the  smoke  charge  out  cool 
quickly  and  are  dissipated  upon  reaching  the 
atmosphere.  The  calcium  carbonate  also  serves  to 
lessen  the  chances  of  fire  by  acting  as  a  chemi¬ 


cal  fire  extinguisher.  The  FBS  unit  is  doubly 
shielded  against  fire  hazard  by  confining  the  hot 
gases  evolved  during  ignition  to  the  inner  Magi¬ 
cube  plastic  housing  and  the  outer  flash  housing 
of  the  feasibility  model  projectile  fuselage  (to 
be  discussed). 

Empirical  Findings  About  the  FBS  Unit 

Many  firings  of  the  feasibility  model  FBS 
unit  were  made  both  in  the  projectile  and  on  a 
test  stand.  Each  time  an  FBS  unit  was  ignited, 
it  was  rated  as  to  its  effectiveness  as  a  cue. 
The  flash  intensity  is  sufficient  to  be  seen  at  a 
200  meter  range  in  bright  sunlight.  The  flash  of 
the  FBS  unit  is  striking  in  low  light  level  con¬ 
ditions  such  as  night,  early  evening  or  an 
extremely  overcast  day;  in  very  bright  sunlight 
conditions,  a  projectile  landing  some  distance 
from  an  individual  would  not  necessarily  catch 
his  attention  via  the  flash  unless  it  happened  to 
fall  within  his  direct  field  of  vision. 

The  deployed  smoke  cloud  produced  by  the 
ejected  calcium  carbonate  is  quite  noticeable  at 
a  200  meter  range  immediately  after  it  is  de¬ 
ployed  (while  the  cloud  has  a  cross  sectional 
dimension  of  1  square  meter).  Under  calm  wind 
conditions,  the  cloud  is  still  entirely  visible 
and  stands  out  against  its  background  when  it  has 
bloomed  to  a  two  square  meter  cross  sectional 
area.  This  cross  section  is  about  maximum  for 
the  "cold  smoke"  cloud.  Beyond  this  cross  sec- 


108 


tlonal  area,  the  oloud  begins  to  fade.  In  addi¬ 
tion,  wind  will  tend  to  disperse  the  oloud,  par¬ 
ticularly  if  the  wind  is  turbulent.  The  oloud 
produced  by  the  feasibility  model  FBS  unit  should 
be  entirely  useable  by  forward  observers  with  a 
clear  field  of  view  up  to  300  meters.  Larger  and 
more  visible  smoke  clouds  oan  be  produoed  by  oom- 
bustion  techniques  within  the  same  projeotile 
delivery  system,  but  are  less  player  safe  and 
present  a  greater  fire  hazard. 

The  sound  level  of  the  FBS  unit  is  on  the 
order  of  108  dB  (measured  3  meters  from  the  point 
of  Impact).  At  a  range  of  200  meters,  the  FBS 
unit  is  dearly  audible,  though  at  this  range  the 
presence  of  any  ambient  noise  in  close  proximity 
to  the  player  (e.g.,  truck  motor,  gunfire,  dose 
talking)  may  totally  obscure  the  sound  signature 
of  the  FBS  unit. 

The  deployment  of  an  FBS  unit  at  a  range  of 
200  meters  is  adequate  to  cue  a  player  engaged  in 
a  war  game  to  the  presenoe  of  incoming  indireot 
fire.  The  effect  is  greatly  enhanced  if  the  cues 
are  dropping  within  100  meters  of  the  players. 
Certain  features  in  the  FBS  unit  could  be  im¬ 
proved  if  the  man  safety  requirements  were  to  be 
less  restrictive.  A  smoke  cloud  visible  at  2 
kilometers  is  well  within  reason  if  a  pyrotechnic 
rather  than  cold  ejected  smoke  could  be  used.  A 
bang  in  excess  of  108  dB  is  easily  achieveable, 
and  would  result  in  a  more  noticeable  cue  for 
projectiles  dropped  at  maximum  range  from  a  war 
game  player.  The  flash  intensity  is  the  one  item 
that  is  difficult  to  increase  safely.  The  safety 
concern  is  not  one  of  man  safety  but  of  fire 
safety.  Since  the  eye  has  a  logarithmic  response 
merely  doubling  the  flash  intensity  does  not  have 
a  profound  effect.  The  flash  intensity  would 
have  to  be  increased  by  a  factor  of  eight  or 
sixteen  to  make  a  significant  difference.  A 
pyrotechnic  flash  would  have  to  be  used  to 
achieve  this  level  of  intensity  with  the  current 
feasibility  model  projectile.  Some  question 
exists  as  to  whether  the  hot  gases  associated 
with  such  a  flash  could  be  easily  contained  with¬ 
in  the  flash  housing  of  the  projectile. 

PROJECTILE 

The  basic  design  of  the  projectile  for  the 
indirect  fire  3imulator/cue  depended  upon  two 
performance  factors;  flight  stability  and  man 
safety  from  the  standpoint  of  impact  blunt  trauma 
to  an  individual.  Tests  were  performed  on  a 
baseline  projectile  to  determine  its  flight  char¬ 
acteristics.  This  baseline  projectile  was  a 
simple  cylindrical  object  with  an  ogive  nose 
section.  Wind  tunnel  tests  were  used  to  deter¬ 
mine  the  stability  of  the  projectile  in  terms  of 
roll,  pitch,  yaw,  and  effects  of  drag  in  the  air 
mass. 

Man  safety  of  the  projeotile  impact  Is  di¬ 
rectly  related  to  impact  momentum.  Impact  momen¬ 
tum  (P)  is  a  function  of  impact  velocity  (V)  and 
projectile  mass  (M). 

P  =  MV 

One  must  reduce  either  the  impact  velocity,  the 
projectile  mi.33,  or  both  to  reduce  the  impact 
momentum. 


A  possible  soheme  to  reduoe  the  impaat  velo¬ 
city  is  to  use  a  variable  drag  technique,  which 
employs  air  brakes  that  are  deployed  late  into 
the  flight  of  the  projeotile  so  as  to  achieve 
maximum  range  with  a  low  drag  profile  and  then, 
at  a  predetermined  point  in  the  flight,  increase 
the  drag  to  significantly  reduoe  projeotile  velo¬ 
city. 

Another  method  to  reaoh  maximum  range  with 
minimum  impact  momentum  is  through  the  use  of  a 
variable  mass  projectile.  A  variable  mass  pro¬ 
jeotile  oan  Jettison  mass  during  flight.  A  con¬ 
venient  source  of  discardable  mass  is  fluid; 
however,  powders  and  even  gases  oan  be  allowed  to 
escape.  One  possible  mode  of  operation  Involves 
the  use  of  a  pressurized  gas  compartment  and  a 
fluid  filled  compartment  within  the  projectile 
which  are  separated  by  a  flexible  balloon-like 
diaphram.  The  fluid  compartment  is  vented  to  the 
outside  through  a  narrow  tube  leading  to  the  aft 
portion  of  the  projeotile.  Upon  launoh,  fluid  is 
forced  through  the  orifice  at  the  end  of  Lite 
projeotile  by  the  expanding  pressurized  gas  oom- 
partment  acting  through  the  flexible  diaphram. 
Mass  is  therefore  continually  lost  by  the  projeo¬ 
tile  throughout  the  flight.  By  careful  timing  of 
the  fluid  release,  the  projectile  can  be  made  to 
achieve  the  maximum  desired  range  by  the  time  the 
entire  fluid  charge  has  been  expended.  The  pro¬ 
jectile  then  falls  to  the  ground  with  a  mass  that 
is  significantly  less  than  the  launch  mass, 
thereby  imparting  lea-  momentum  to  any  object 
that  it  strikes.  Combinations  of  variable  mass 
and  variable  drag  are  a? so  possible.  Both  the 
variable  mass  and  variable  drag  concepts  are 
valid  methods  for  reducing  impact  momentum,  but 
the  complexity  and  cost  of  the  projectile  is 
increased,  wnile  the  timing  of  these  final  momen¬ 
tum-reducing  schemes  introduces  an  additional 
source  of  error  into  the  launch -system. 

A  constant  mass  projectile  is  one  having  a 
final  impact  mass  that  is  the  same  as  its  launch 
mass,  and  a  final  velocity  that  is  proportional 
to  its  launch  velooity  (where  the  constant  of 
proportionality  relates  to  the  aerodynamic  drag 
coefficient  of  the  projectile  shape).  In  any  of 
the  projectile  configurations  mentioned,  terminal 
momentum  must  be  low  enough  to  allow  impact  upon 
an  individual  without  causing  bodily  harm.  (Note 
however  that  harm  to  an  individual  is  possible  by 
any  practical  projectile  regardless  of  configura¬ 
tion  if  the  Impact  is  sustained  upon  certain 
areas  of  the  body  (e.g.,  eyes)).  One  major  ad¬ 
vantage  of  a  constant  mas3  projectile  arises  from 
its  inhfe.  jnt  reliability.  Both  variable  mass  and 
variable  drag  schemes  could  fail  to  deploy  their 
final  momentum-reducing  mechanisms  after  launch, 
resulting  in  an  unsafe  impact  momentum.  Such  a 
condition  cannot  occur  with  a  man-safe  constant 
mass  projectile. 

Constant  Mass  Final  Velocity  Tests  and  Man  Safety 

Early  in  the  development  of  the  pneumatic 
launcher,  reuseable  constant  mass  non-variable 
drag  balsa  wood  test  projectiles  were  used  to 
test  the  launcher.  These  projectiles  weighed 
anywhere  from  two  to  five  ounces.  When  conduct¬ 
ing  tests  at  the  150  to  200  meter  rarge,  techni¬ 
cians  standing  nearby  noted  that  the  impact  velo- 
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city  of  the  projectile  appeared  sufficiently  slow 
such  that  they  were  willing  to  try  to  catch  them 
in  midflight.  Final  velocity  tests  were  then 
conducted  to  determine  the  man  safety  of  the 
projectile  if  neither  variable  mass  nor  variable 
drag  techniques  were  employed. 

Three  basic  tests  were  performed  to  assess 
the  final  velocity  of  the  projectiles.  First,  a 
projectile  with  a  calibrated  momentum  sensor  was 
used  to  measure  the  terminal  momentum  and  hence 
velocity  in  situ.  A  second  method  involved  high 
speed  photography  of  the  projectile  upon  impact, 
and  the  third,  a  standard  police  radar  was  used 
to  measure  the  velocity  Just  prior  to  impact. 

All  three  methods  yielded  corresponding 
final  velocity  information.  For  the  feasibility 
model  projectile,  the  measured  final  velocity  was 
on  the  order  of  64  kph  upon  imp  'it.  Other  ob¬ 
jects  wore  investigated  that  might  also  have  a  64 
kph  impact  velocity  to  obtain  a  feel  for  the 
damage  that  might  be  incurred  by  a  human  struck 
by  such  a  projectile.  In  particular,  a  tennis 


serve  wa3  studied  beoause  the  weight  of  a  tennis 
ball  was  within  grams  of  the  feasibility  model 
projectile,  and  therefore,  would  be  a  good  indi¬ 
cator.  Using  the  polios  radar,  a  tennis  ball  was 
served  numerous  times  directly  at  the  radar  an¬ 
tenna.  Spectrum  analysis  showed  that  the  tennis 
serve  also  yielded  a  velocity  of  approximately  64 
kph.  This  means  that  the  danger  of  human  damage 
due  to  a  strike  by  the  feasibility  model  projec¬ 
tile  would  correspond  to  that  expected  of  a 
strike  by  a  tennis  ball  being  served.  Various 
other  sports  activities  were  found  to  involve 
greater  danger  of  damage  due  to  strikes  by  the 
playing  implements.  For  example,  a  fly  baseball 
presents  a  greater  danger  to  an  outfielder  than 
does  the  feasibility  model  projectile. 

After  reducing  the  impact  momentum  data  and 
finding  that  the  final  velocity  was  64  kph  for 
the  feasibility  model  projectile,  the  technicians 
were  allowed  to  attempt  to  catch  dummy  projec¬ 
tiles  in  flight.  The  picture  sequence  in  Figure 
2  shows  one  such  successful  catch. 


c 


Figure  2.  Impact  Safety  Demonstration 
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Thg  Feasibility  Model  Constant  Hers  Projeotile 

Once  it  was  demonstrated  that  projeotiloa 
war*  relatively  impact  safa  to  humans,  a  daoision 
was  made  that  variabla  drag  and  variable  mass 
teohniques  would  not  be  neoeasary  to  achieve  the 
oontraot  goals  since  constant  mass  projectiles 
could  be  launohed  to  ranges  in  exoess  of  200 
meters  and  still  be  nan  safe  from  an  impact 
standpoint. 

Figure  3  shows  the  components  used  to  oon- 
struot  the  feasibility  model  projeotile.  These 
oomponents  inolude  a  polystyrene  tail  section, 
rubber  hemispherial  nose  seotion,  two  piece  epoxy 
arming  meohanism  and  guide  meohanism,  and  Lexan 
flash  housing.  Eaoh  of  these  oomponents  was 
oarefully  designed  as  to  size  and  weight  to  re¬ 
sult  in  a  feasibility  model  projeotile  of  precise 
center  of  gravity  and  mass.  The  tall  seotion  is 
made  from  fused  polystyrene  beads.  A  smooth 
finish  and  a  rough  finish  is  possible  depending 
on  the  size  beads  used  in  the  manufacturing. 
Polystyrene  beads  that  will  result  in  a  rough 
finish  for  the  tail  section  are  necessary  beoause 
minor  turbulanoe  is  formed  over  the  surfaoe  of 
the  tail  section  to  break  the  laminar  air  flow  in 
muoh  the  same  way  that  the  dimples  on  a  golfball 
provide  stability  by  breaking  the  laminar  flow. 


The  minor  turbulanoe  oausas  an  lnoreaaed  drag  in 
the  tall  section  whioh  results  in  projeotile 
stability  by  assuring  a  greater  side  drag  behind 
the  oenter  of  gravity  than  in  front.  All  por¬ 
tions  of  the  projeotile  from  the  tail  section 
forward  must  be  kept  as  smooth  as  possible  for 
the  stabilising  effeot  or  the  rough  tail  seotion 
to  be  effaotive. 

The  Lexan  flash  housing  is  made  of  10  mil 
Lexan  tubing.  Lexan  was  chosen  as  a  flash  hous¬ 
ing  material  beoause  it  is  self-extinguishing  and 
does  not  shatter  during  impact.  The  uoe  of  Lexan 
assures  that  there  will  be  no  shrapnel  upon  im¬ 
pact  and  also  prevents  the  high  temperature  gases 
evolved  within  the  FBS  unit  from  burning  through 
to  the  outside. 

The  arming  meohanism  has  four  firing  pins 
whioh  are  foroed  up  through  the  base  of  the  FBS 
unit  upon  lmpaot,  and  result  in  FBS  unit  igni¬ 
tion.  In  the  unarmed  position,  these  pins  are 
physlaally  misaligned  with  the  FBS  unit  ignition 
system.  When  plaoed  in  the  armed  position  by 
manually  rotating  the  arming  mechanism  relative 
to  the  projeotile  fuselage,  the  pins  are  aligned 
with  holes  in  the  base  of  the  FBS  unit,  thereby 
allowing  a  forward  lmpaot  to  foroe  the  firing 
pins  into  the  holes  oauslng  FBS  unit  detona- 


Flgure  3.  Projectile  Assembly 
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tion.  Tests  have  shown  that  an  unarmed  projeo- 
tile  oan  be  handled  rather  roughly  and  evan  drop¬ 
ped  without  danger  of  Igniting  the  PBS  unit;  how¬ 
ever,  when  armed,  dropping  of  the  projectile  on 
the  nose  from  a  height  of  as  little  as  1  foot  oan 
set  off  the  PBS  unit. 

The  nose  oone  of  the  feasibility  model  pro¬ 
jectile  is  hemispherical,  and  is  made  of  a  foam 
rubber  compound.  This  nose  oone  adds  the  weight 
neoessary  to  bring  the  oenter  of  gravity  to  the 
very  front  of  the  flash  housing.  From  an  aero¬ 
dynamic  standpoint  the  hemispherical  nose  shape 
is  a  731  drag  improvement  over  a  simple  flat 
ended  projeotlle  nose.  A  fully  assembled  feasi¬ 
bility  model  projectile  is  shown  in  Figure 


Figure  *).  Final  Feasibility  Model 
Projectile  With  FBS  Unit 


PNEUMATIC  LAUNCHER 

Early  attempts  to  test  a  pneumatic  launoher 
employed  a  freon  tank,  a  two  inch  brass  ball 
valve,  and  a  length  of  Polyvinal  Chloride  (PVC) 
plastic  pipe,  as  shown  in  Figure  5a.  Figures  5b 
through  5d  show  the  pneumatic  launoher  that  was 
ultimately  developed.  Figure  6  identifies  the 
major  components  which  comprise  this  feasibility 
model  pneumatio  launoher.  It  operates  on  the 
pneumatic  principle  of  a  blow  gun  using  vaporized 
liquid  C02  as  a  propellant.  An  explosive  flapper 
valve  is  used  to  transfer  pressurized  C02  gas 
from  an  intermediate  holding  reservoir  to  the 
barrel  of  the  launoher  in  whioh  a  projectile  has 
been  placed.  The  expanding  C02  gas  is  capable  of 
ejeoting  a  two  to  three  ounce  projectile  from  the 
barrel  and  hurling  it  in  exoess  of  200  meters 
using  C02  reservoir  pressures  as  low  as  30  psig 
(see  Figure  7). 

Valve  Considerations  and  Pressure  Rise  Time 
Within  the  Barrel 

The  early  pneumatic  launoher  used  a  large 
diameter  ball  valve  which  was  activated  through  a 
spring  mechanism  to  achieve  as  fast  an  opening  as 
possible.  After  extensive  testing,  data  showed 
that  at  higher  pressures  increased  range  was  not 
appreciable.  Analysis  indicated  that  the  valve 
opening  speed  played  an  important  role  in  achiev¬ 
ing  maximum  range  at  these  higher  pressures  due 
to  the  pressure  rise  time  within  the  barrel. 
When  the  valve  is  initially  opened,  the  pressure 


in  the  reservoir  begins  to  drop,  and  the  pressure 
within  the  barrel  behind  the  projeotlle  begins  to 
rise.  Immediately  upon  overcoming  static  fric¬ 
tion,  the  projeotlle  moves  down  the  barrel  in 
front  of  the  lnoreaslng  pressure  wavo  front. 
Tests  involving  the  ball  valve  anu  high  pres¬ 
sures,  demonstrated  that  the  projeotlle  oould 
move  down  the  barrel  and,  in  fact,  leave  the 
barrel  before  the  complete  build-up  of  barrel 
pressure  had  taken  plaoe.  As  reservoir  pressure 
was  increased  beyond  this  point,  it  had  little 
effeot  because  the  pressure  front  oould  not  fully 
transfer  its  energy  to  the  projeotlle  onoe  the 
projeotlle  had  oleared  the  end  of  the  barrel. 

An  improved  valve  design  emerged  wherein  a 
flapper  valve  was  used  to  explosively  transfer 
pressure  from  the  reservoir  to  the  barrel.  A 
significant  Increase  in  performance  was  immedi¬ 
ately  noticed.  By  using  the  pressure  in  the  re¬ 
servoir  to  blast  the  valve  open,  it  was  possible 
to  obtain  a  reservolr-to-barrel  pressure  transfer 
in  much  less  time  than  previously  achievable  with 
the  ball  valve/spring  arrangement.  The  pressure 
in  the  barrel  oould  reaoh  a  maximum  before  the 
projeotlle  left  the  end  of  the  barrel;  therefore, 
up  to  30  psig,  most  of  the  pressure  stored  in  the 
reservoir  oould  be  applied  to  the  projeotlle.  If 
pressure  were  to  be  inoreased  beyond  30  psig  a 
limit  would  be  readied  where  the  reservoir  pres¬ 
sure  transfer  time  would  exoeed  the  projectile 
time-in-barrel.  Other  higher  speed  valve  con¬ 
figurations,  suoh  as  exploding  diaphrams,  are 
possible;  however,  these  devioes  are  not  reusable 
and  would  not  be  suitable  for  automated  operation 
of  the  launcher. 

Muzzle  Brake 

A  muzzle  brake  was  designed  for  use  with  the 
pneumatic  launoher.  The  purpose  of  this  break 
was  to  minimize  the  effect  of  diffracting  air 
currents  passing  from  the  barrel  into  the  atmos¬ 
phere.  If  allowed  to  go  unchecked,  these  dif¬ 
fracting  air  currents  can  defleot  the  tail  of  the 
projectile  as  it  clears  the  end  of  the  barrel 
(high  speed  motion  pictures  of  test  projectiles 
leaving  the  barrel  without  the  muzzle  brake  have 
visually  confirmed  this  deflection).  These  ini¬ 
tial  tail  deflections  result  in  undesired  trajec¬ 
tory  perturbations.  Placement  of  parallel  plates 
at  the  end  of  the  barrel  wnich  have  holes  that 
are  slightly  larger  than  the  inner  barrel  dia¬ 
meter  passing  through  the  oenter  of  each  plate, 
allows  the  projectile  to  move  from  the  barrel, 
through  the  plates,  and  into  the  atmosphere  under 
the  direct  force  of  the  planar  pressure  front 
that  drives  the  projectile  up  the  barrel.  Any 
pressure  wave  fronts  which  are  off-axis  impinge 
upon  the  parallel  plates  and  are  redirected  per¬ 
pendicular  to  the  flow  direction  of  the  main  wave 
front.  Several  plates  were  employed  to  increase 
the  efficiency  of  the  muzzle  brake  (see  Figure 
5e).  Photographic,  CEP,  and  range  data  analyses 
all  confirmed  the  effectiveness  of  the  muzzle 
brake. 

The  Creation  of  the  Air  Bearing 

During  launch,  the  pressure  front  formed 
behind  the  projectile  forces  it  down  the  bar¬ 
rel.  Initially,  there  is  contact  between  the 
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Figure  5.  Pneumatic  Launcher 


projectile  and  the  barrel.  ks  the  projectile 
begins  to  move  down  the  barrel,  however,  it  rides 
on  a  cushion  of  air  as  the  pressure  from  behind 
escapes  along  the  narrow  projeatile/barrel-wall 
interface.  This  air  bearing  is  not  formed  at  low 
launch  pressures  (e.g.,  five  psig).  Only  when 
the  launch  pressure  is  of  sufficient  magnitude  to 
force  ga3  past  the  projectile,  does  the  air  bear¬ 
ing  form.  The  movement  of  the  projectile  can 
also  enhance  the  formation  of  the  air  bearing. 
There  is  a  point  at  which  the  projectile  will 
begin  to  move  up  the  barrel  at  low  pressures 
(after  having  overcome  static  friction)  while 
maintaining  significant  barrel  contact.  After 
obtaining  a  certain  barrel  velocity  at  these  low 
pressures,  leakage  occurs  around  the  projectile 
which  eventually  forms  an  air  bearing.  The  pro¬ 
duction  of  the  air  bearing  is  essential  for  ef¬ 


ficient  operation  of  the  pneumatic  launcher. 
Best  results  are  achieved  at  pressures  above  8 
psig  due  to  the  formation  of  the  air  bearing. 
Short  range  launches  are  therefore  best  aohieved 
through  the  increase  of  quadrant  elevation, 
rather  than  the  continual  decrease  in  reservoir 
pressure. 

Wide  Area  Coverage  Considerations 

Consideration  was  given  to  the  question  of 
maximum  range  attainable  versus  launcher  cost. 
Either  a  single  launcher  must  be  capable  of  360° 
operation  at  a  range  sufficient  to  cover  an  area, 
or  a  number  of  shorter  range  360°-operable 
launchers  must  be  employed  in  a  matrix  to  achieve 
maximum  coverage  of  a  given  area.  Figure  8  shows 
the  geometry  used  to  determine  the  spacing  of 
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Figure  6,  Launcher  Assembly 


Figure  7.  Range  Performance  vs.  Reservoir  Pressure 
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launchers  of  a  maximum  launch  range  (R)  within 
the  matrix.  The  separation  (D)  necessary  for 
these  launchers  is  related  to  the  maximum  range 
of  the  launcher  by  the  curve  shown  in  the  graph 
of  Figure  8.  Note  that  this  curve  is  exponential 
and  that  the  average  number  of  launchers  de¬ 
creases  drastically  as  range  capability  is  in¬ 
creased.  Currently  the  feasibility  model  launch¬ 
er  can  attain  a  maximum  range  of  approximately 
200  meters  usinf  reservoir  pressures  that  do  not 
exceed  30  psig.  Additional  research  will  be 
necessary  to  determine  if  significantly  greater 
ranges  can  be  achieved  through  either  launcher 
modification  (e.g.,  increased  reservoir  pressure 
or  advanced  valve  design)  or  modification  of  the 
feasibility  model  projectile  (computer  simula¬ 
tions  have  recently  shown  that  maximum  launch 
ranges  of  500  meters  (1  km  circular  coverage)  are 
possible  using  increased  launch  pressures  coupled 
with  a  reduced  diameter  projectile  that  has  a 
launch  weight  two  and  one  half  times  greater  than 
that  of  the  feasibility  model  projectile.  Veri¬ 
fication  of  these  computer  simulated  results  is 
forthcoming).  A  performance  constraint  is  placed 
on  the  maximum  achievable  range  however.  As  the 


maximum  range  capability  of  the  launcher  is  in¬ 
creased,  so  is  the  projectile  tioe-of-f light. 
The  projectile  trajectory  is  subject  to  wind- 
induced  perturbations  as  long  as  the  projectile 
is  in  flight,  and  therefore  the  CEP  of  the  in¬ 
direct  fire  cueing  system  will  degrade  with  in¬ 
creased  range  capability. 

Man-Safety  Aspects  of  the  Launcher 

Since  the  launcher  normally  achieves  muzzle 
velocities  on  the  order  of  145  to  160  kph  (see 
Figure  9),  oare  should  be  taken  to  avoid  direct 
impact  by  the  projectile  at  point  blank  range 
since  the  projectile  is  moving  at  its  highest 
velocity  immediately  after  leaving  the  barrel  of 
the  launcher.  Use  of  the  feasibility  model  re¬ 
quires  only  that  operators  and  onlookers  remain 
out  of  the  field  directly  in  front  of  the  muz¬ 
zle.  All  other  aspects  of  the  launcher  are  man- 
safe.  The  propellant  used  is  carbon  dioxide,  and 
the  amount  expelled  per  3hot  is  not  significant 
when  expulsion  is  into  the  open  atmosphere.  The 
sound  level  of  a  launch  is  about  98  dB  as  mea¬ 
sured  three  meters  in  front  of  the  launcher  and 
is  well  within  the  Surgeon  General's  guidelines. 


Figure  8.  Number  of  Simulators  vs.  Maximum  Simulator  Range 
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Figure  9.  Muzzle  Velocity  vs.  Reservoir  Pressure 


OVERALL  SYSTEM  PERFORMANCE 

The  performance  of  the  system  was  ultimately 
determined  through  a  number  of  experiments  in  the 
field  which  were  geared  toward  measuring  the  ac¬ 
curacy  with  which  a  cue  could  be  successfully 
delivered  to  a  predetermined  location  and  deto¬ 
nated  upon  impact.  The  accuracy  of  ballistic 
objects  is  usually  described  by  the  elliptical 
probable  error  (EPE)  of  their  impact  points  about 
their  corporate  controid.  For  convenience,  the 
major  and  minor  axes  of  the  typically  elliptical 
impact  pattern  are  normalized  to  yield  a  circle 
of  the  same  area  as  the  original  ellipse  which  is 
called  the  circular  error  probable  (CEP). 

Circular  error  probable  is  defined  as  the 
radial  distance  from  the  center  of  impact  which 
is  as  likely  to  be  exceeded  as  not.  This  means 
that  it  is  a  circle  whose  center  is  at  the  impact 
area  centroid  and  includes  501  of  all  the  points 
of  impact.  The  radius  of  this  circle  equals  1 
CEP  (1). 

Circular  error  probable  tests  were  conducted 
on  the  feasibility  model  system  by  firing  a  se¬ 
quence  of  shots  without  changing  QE  or  reservoir 
pressure  from  shot  to  shot.  A  theodolite  mounted 
downrange  from  the  launcher  was  used  to  accu¬ 
rately  plot  the  relative  angle  of  each  impact. 
In  addition,  a  measurement  of  the  impact  distance 
from  the  theodolite  was  made.  Wind  direction  and 
velocity  were  simultaneously  measured  (at  a 
height  of  10  meters)  to  assure  that  the  shots 
were  occuring  under  minimal  wind  conditions. 


The  measurement  of  each  impact  point  yielded 
polar  information  which  was  transferred  to  set  of 
cartesian  coordinates  a3  shown  in  the  graph  of 
Figure  10.  The  CEP  of  17.35  meters  derived  for 
these  tests  indicates  performance  far  exceeding 
that  required  by  the  Army  (25  m  requirement).  In 
fact  100)1  of  the  impacts  fell  within  the  25  meter 
required  CEP. 

The  effects  of  upper  level  winds  on  the  tra¬ 
jectory  can  be  severe.  An  upper  level  wind  shear 
would,  on  numerous  occasions,  cause  the  projec¬ 
tile  to  move  significantly  off  course.  These 
upper  level  winds  are  difficult  to  predict  over 
the  entire  test  range.  The  CEP  impact  graph  of 
Figure  10  was  constructed  from  data  taken  during 
the  final  acceptance  tests.  On  other  occasslons, 
similar  tests  yielded  CEP's  as  ,  low  as  6.1  meters 
which  is  likely  due  to  differences  in  upper  level 
wind  turbulance.  *■ 

A  projectile  fire  hazard  assessment  was  also 
conducted  during  the  final  acceptance  tests 
wherein  a  standard  projectile  was  ignited  in  the 
presence  of  gasoline  saturated  paper.  The  pro¬ 
jectile  was  ignited  successfully,  deploying  its 
FBS  unit  without  igniting  the  surrounding  gaso¬ 
line  saturated  papers  or  the  gasoline  vapor  in 
the  air.  Recognition  must  be  given  to  the  fact 
that  this  test  is  not  conclusive  proof  that  the 
feasibility  model  projectile  would  not  cause 
grass  fires  under  normal  use.  However,  the  fact 
that  the  gasoline  saturated  paper  (considered  to 
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Figure  10,  CEP  Acceptance  Test  Results 
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be  more  flammable  than  dry  grass)  did  not  ignite 
indicates  that  the  projectile  can  be  deployed  in 
the  presence  of  highly  flammable  material  without 
necessarily  igniting  that  material. 

Another  set  of  measurements  conducted  during 
the  final  acceptance  test  consisted  of  maximum 
range  tests.  These  tests  were  designed  to  eval¬ 
uate  the  maximum  achievable  range  of  the  feasibi¬ 
lity  model  system.  Projectiles  launched  with 
pressures  of  25  psig  and  launcher  QEs  of  H5  de¬ 
grees  had  impact  points  in  excess  of  200  meters 
from  the  launch  site.  These  projectiles  ignited 
correctly  upon  impact  deploying  a  visible  smoke 
cloud  and  audible  bang  at  the  200+  meter  ranges. 

CONCLUSIONS 

FBS  Unit  Determinations 


Many  ignition  techniques,  sound  generators, 
smcke  generators,  and  light  sources  were  investi¬ 
gated  for  use  in  the  FBS  unit.  The  FBS  unit  was 
developed  around  a  Magicube  primer  and  a  shotgun 
shell  loaded  with  a  cold  particulate  smoke  gen¬ 
erator.  Pyrotechnic  generators  were  found  to  be 
more  effective  cues  than  cold  particulate  smokes; 
however,  they  were  less  man-safe  and  a  greater 
fire  hazard.  The  sound  level  of  the  shotgun 
shell  approach  is  easily  adjustable  and  lies 
within  the  acceptable  ranges  as  set  forth  by  the 
Surgeon  General  of  the  United  States.  The  light 
output  from  the  Magicube  is  very  intense  but 
brief.  The  use  of  the  Magicube  as  a  primer  for 
the  shotgun  shell  yields  a  small  lightweight 
package  that  is  highly  stable  and  man-safe. 
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Projectile  Determinations 

Variable  mass,  variable  drag,  constant  mass, 
and  hybrid  configurations  of  projectiles  were 
investigated.  A  baseline  projectile  was  hypothe¬ 
sized  and  modified  numerous  times  to  obtain  the 
feasibility  model  projectile.  These  modifies-  5 

tions  were  based  on  simulations  by  both  computer  f 

and  direct  wind  tunnel  experiments,  in  addition  ■ 

to  empirical  data  derived  from  actual  field  >  J 

tests.  j 

I 

Flight  stability  is  a  function  of  two  major 
factors.  First,  a  hemispherical  nose  cone  is  s- 

used  which  decreases  the  forward  drag.  Second,  a  { 

rough  finish  is  used  on  the  tail  section  to 
create  minor  turbulence  which  causes  increased 
drag  in  the  tail  section  resulting  in  projectile 
stability  by  assuring  a  greater  side  drag  behind 
the  center  of  gravity  than  in  front.  All  por¬ 
tions  of  the  projectile,  from  the  tail  section 
forward,  must  therefore  be  kept  as  smooth  as 
possible  for  the  stabilizing  effects  of  the  rough 
tail  section  to  be  effective. 

Launcher  Determinations  i  ] 

Several  launcher  schemes  were  investigated  i 

during  the  indirect  fire  simulation/cueing  pro-  I  f 

gram.  Of  those  investigated  three  were  imple-  i  .] 

mented;  of  these  three,  the  pneumatic  launcher  ■  i 

was  chosen  for  development.  j 

A  fast  acting  valve  was  found  to  be  essen-  >  j 

tial  to  efficient  operation  of  the  pneumatic  !  j 
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launcher.  The  valve  design  chosen  was  an  explo¬ 
sive  flapper  valve. 

Barrel  length  also  affects  launcher  effi¬ 
ciency.  Of  those  tested,  the  four  foot  barrel 
length  was  chosen  because  of  its  tractability . 
As  pressures  are  increased  in  later  prototype 
models,  the  barrel  should  be  increased  in  length. 

During  the  course  of  pneumatic  launcher 
evaluation,  an  air  bearing  was  fourd  to  be 
created  between  the  projectile  and  the  barrel 
wall  for  pressures  above  8  psig.  The  creation  of 
the  air  bearing  is  essential  to  efficient  opera¬ 
tion  and  maximum  range. 

Projectile  fish-tailing  immediately  upon 
launch  lead  to  the  development  of  a  muzzle  brake 
which  channeled  off-axis  gas  flow  away  from  the 
tail  section  of  the  newly  airborne  projectile. 
The  addition  of  the  muzzle  brake  improved  impact 
groupings. 

Extensive  testing  indicates  that  upper  level 
winds  are  a  dominant  factor  in  biasing  impact 
centroids.  QEs  of  less  than  60  degrees  were 
found  to  be  desirable  in  order  to  avoid  these 
upper  level  winds.  Observance  of  the  60  degree 
limit  becomes  more  important  in  future  prototype 
launchers  where  maximum  ranges  will  be  Increased. 

Production  Conclusions 

All  expendable  components  of  the  feasibility 
model  system  are  designed  to  be  conducive  to  mass 
production  techniques.  Whenever  possible,  com¬ 
ponents  are  cast  or  molded  from  specific  types  of 
plastics.  The  choice  of  plastic  for  use  in  a 
given  component  is  dictated  by  its  weight,  tinsel 
strength,  or  elasticity.  The  tail  sections  are 
made  from  expanded  polystyrene  because  of  its 
extremely  low  density.  The  arming  device  is  made 
from  two  types  of  plastic;  one  being  very  rigid 
and  the  other  being  elastic.  Rigidity  was  impor¬ 
tant  in  the  Magicube  receiver  section  of  the 
arming  device  since  this  section  provided  the 
major  structural  strength  for  the  front  half 
of  the  flash  housing.  The  upper  section  of  the 
arming  device  contains  machined  plastic  leaf 
springs  which  mu3t  be  able  to  flex  without  break¬ 
ing,  so  a  different,  more  elastic  kind  of  plas¬ 
tic  was  nece33ary  to  implement  this  component. 
Other  parts  of  the  system  must  be  fire  proof  or 
self -extinguishing.  The  flash  housing  is  one 
example,  being  made  out  of  Lexan,  a  self-extin- 
guishing  polycarbonate  material.  The  nose 
section  mu3t  be  able  to  maintain  its  hemispher¬ 
ical  3hape  under  the  acceleration  of  launch,  but 
also  must  be  able  to  deform  upon  impact  with  a 
human  to  increase  the  level  of  man-safeness  of 
the  projectile.  For  this  application,  a  cast 
foam  rubber  compound  was  employed. 


Man-Safety  Inferences 


The  feasibility  model  projectile  was  demon¬ 
strated  to  be  impact  safe  for  individuals  in 
visual  contact  with  the  incoming  round.  Fully 
outfitted  soldiers  engaged  in  war  games  should  be 
as  sate  from  a  direct  impact  given  that  the  im¬ 
pact  does  not  occur  on  the  eye  or  in  general,  the 
facial  region.  Adequate  eye  protection  would 


effectively  render  a  facial  impact  harmless.  The 
sound  and  light  level  outputs  from  the  FBS  unit 
are  within  acceptable  medical  standards.  The 
cold  particulate  smoke  is  soluble  in  the  lungs 
and  is  non-toxic  in  the  quantities  to  be  encoun¬ 
tered  during  an  actual  war  game  engagement. 

General  Conclusion 

Extensive  experimentation  has  shown  that  the 
feasibility  model  system  performs  in  accordance 
with  theory  and  meets  or  exceeds  all  contract 
requirements.  The  feasibility  model  system,  as 
delivered,  proves  the  indirect  fire  cueing  system 
concept  to  be  valid.  Further  research  is  neces¬ 
sary  to  improve  upon  this  system,  however. 
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ABSTRACT 

Computer  Controlled  Trainer  Simulation  traditionally  has  been  done  with  minicomputer 
class  processors.  Instruction,  speed  and  flexibility,  as  well  as  good  hardware  and  soft¬ 
ware  support,  are  some  of  the  reasons  for  the  domination  of  the  minicomputer  in  the  area 
of  simulation.  However,  over  the  past  decade,  microprocessor  performance  has  Improved  to 
such  an  extent  that  microprocessors  may  rival  some  of  the  best  minicomputers  currently 
available.  Microprocessors  may  be  used  to  offload  the  mini  In  areas  such  as  complex 
;  Ml  time  handling  of  I/O  and  number  crunching,  as  well  as  monitoring  hardware  status. 
Distributed  processors  allow  the  mini  to  process  more  data,  with  less  hardware  needed  to 
perform  the  simulation  task.  Microprocessors  may  also  replace  minicomputers  completely 
in  systems  requiring  real  time  number  crunching  with  limited  precision  calculations,  such 
as  radar  simulation  or  part  task  trainers.  Additionally,  software  support  has  become 
quite  extensive  with  many  of  the  leading  microcomputers.  Including  ANSI  FORTRAN,  Basic  and 
Pascal,  making  the  micro  a  viable  candidate  to  replace  the  minicomputer  for  simulation  of 
military  or  commercial  equipment.  Such  a  system  offers  reduction  In  costs,  weight,  and 
power  consumption  with  Increased  reliability  and  flexibility. 


INTRODUCTION 

The  emergence,  in  the  late  50' s  and  early 
60's,  of  the  digital  computer  In  training  devices 
provided  significant  advancements  in  training  cap¬ 
ability.  ReprogrammablUty,  improved  accuracy, 
ability  to  develop  hardware  and  software  independ¬ 
ently  and  ability  to  use  the  digital  computer  to 
test  the  entire  training  device  were  all  favorable 
characteristics  which  led  to  the  evolvement  of  the 
digital  computer  as  the  heart  of  the  modern  train¬ 
ing  device. 

Since  It  was  first  introduced  In  training 
simulation,  improvements  In  the  computer's  abilities 
such  as  more  Instructions,  faster  speed,  and  larger 
word  sizes  have  strengthened  the  role  of  the  digit¬ 
al  computer.  Between  November  '65  and  January  '77 
the  number  of  digital  computers  inventoried  by  NTEC 
rose  from  51  to  587.(1)  However,  framing  time 
crunch  and  discrete  system  anomalies  associated 
with  models  of  analog  systems  continue  to  be  a 
problem.  As  system  complexity  Increases,  it  be¬ 
comes  more  and  more  difficult  for  digital  computers 
to  provide  realistic  outputs  to  the  student  In  real 
time.  Often  enough  there  are  complaints  that  it 
doesn't  "feel",  "look"  or  "sound"  like  the  real 
system.  Microprocessors  have  evolved  to  the  point 
today  where  they  can  perform  many  of  the  tasks 
normally  associated  with  a  minicomputer.  Because 
total  mission  training  constraints  are  requiring 
much  larger  computers  and  because  microprocessors 
can  easily  handle  data  manipulation  and  number 
crunching,  a  marriage  of  microcomputers  and  mini¬ 
computers  In  a  distributed  processing  environment 
is  called  for.  Lower  costs  of  microprocessor  sys¬ 
tems  and  significantly  less  development  time  are 
also  factors  which  make  the  microprocessor  an 
Ideal  addition  to  training  devices.  Additionally, 
the  power  of  the  microprocessor  makes  It  a  viable 
candidate  for  smaller  systems  such  as  radar  simula¬ 
tion  and  part  task  trainers  Instead  of  the  mini¬ 
computer. 

Of  the  587  computers  Inventoried  In  January 
'77  there  were  40  different  languages  In  use. 


Present  requirements  for  standardized  languages, 
such  as  PASCAL,  to  provide  software  transportabil¬ 
ity,  are  hoped  to  maximize  use  of  programs  between 
training  devices  and  computers.  Many  of  the  lead¬ 
ing  microprocessor  manufacturers  offer  development 
systems  and  software  support  packages  which  allow 
programing  in  FORTRAN,  Basic,  Pascal  and  assembly 
languages. 

The  main  objectives  for  microprocessor  utili¬ 
zation  are: 

Peripheral  Processing 

•  scaling  from  engineering  values  to  binary 
values 

•  curve  fitting  for  non-linear  devices 

•  packing  and  unpacking  of  digital  data 

•  self-checking  on-line  background  programs 
and  off-line  automatic  test  capability 

•  state  change  detection  of  input  data 

•  relocatable  I/O 

Stand  Alone  Trainer 

•  complete  standard  trainer  electronic 
package 

•  standard  peripherals 

•  maintenance  trainers,  part  task  trainers. 

The  benefits  derived  from  microprocessor  usage 

are: 

•  production  cost  savings  due  to  fewer  parts 
and  wiring 

•  Increased  system  reliability 
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•  unburdens  main  processor 

•  unburdens  processor  channel 

•  eliminates  need  for  mini  in  small  systems 

•  permits  a  more  rugged  trainer. 

Some  disadvantages  of  micro  based  systems  are: 

•  the  initial  costs  of  the  development  sys¬ 
tem  are  high 

•  lack  of  built-in  control  panel  for  operator 
control 

•  most  16-bit  micros  do  not  support  floating 
point  operations. 

The  advantages  and  disadvantages  have  to  be 
evaluated  up  front  when  applying  microprocessors  to 
training  simulators.  In  the  sections  that  follow, 
it  is  shown  how  the  microprocessor  could  have  pro¬ 
vided  significant  benefits  in  the  development  of 
the  C5A-Cockpit  Procedures  Trainer  (CPT).  The 
design  philosophy  of  using  microprocessors  in  stand 
alone  radar  trainers  is  also  presented.  The  per¬ 
formance  benefits  of  using  a  microprocessor  to 
offload  tasks  from  a  minicomputer  are  shown. 

MICROPROCESSORS  AND  PERIPHERAL  PROCESSING 

Computer  memory  size  requirements  are  contin¬ 
ually  increasing.  Faster  iteration  rates,  quantity 
of  I/O  and  complexity  of  simulated  systems,  such  as 
experienced  in  Weapon  System  Trainers,  all  contri¬ 
bute  to  the  need  for  more  and  more  computer  power. 
Multiple  computer  systems  are  not  uncommon.  Micro¬ 
processor  systems  can  relieve  the  minicomputer  of 
many  of  the  time  consuming  tasks.  The  total  I/O 
time  required  for  Device  A/F-37A-T65,  the  C5A  Cock¬ 
pit  Procedures  Trainer,  averages  18%  of  real  time. 
Digital  data  is  processed  at  a  5-Hz  rate  and  analog 
data  is  processed  at  a  10-Hz  rate  and  uses  a  Harris 
Slash  5  computer  with  an  Automatic  Block  Controller 
(ABC)  for  I/O  transfer. 

The  following  paragraphs  illustrate  some  typ¬ 
ical  I/O  handling  requirements  and  how  they  relate 
to  the  C5A-CPT  I/O  processing  tasks. 

These  tasks  include: 

•  Polling  of  I/O 

•  Packing  and  unpacking  of  discrete  data 

•  Scaling  and  curve  fitting  of  analog  data 

-  Inputting  analog  data 

-  Curve  fitting 

-  Sine/cosine  conversion. 

Polling  of  I/O 

Polling  techniques  for  transferring  data  be¬ 
tween  the  computer  memory  and  the  I/O  equipment  use 
valuable  time,  especially  at  higher  iteration  rates. 
The  central  processor  reserves  a  storage  area  in 
memory  separate  from  the  memory  being  used  for  pro¬ 
cessing.  This  reserve  storage  area  is  required  to 
buffer  all  input  and  output  data.  All  data  being 


transmitted  to  the  I/O  device  is  first  placed  in 
the  buffer  before  being  transmitted.  Similarly  all 
data  received  from  the  I/O  device  is  first  placed 
in  the  storage  area.  In  the  C5-CPT  trainer,  digi¬ 
tal  data  is  transferred  at  5  times  per  second  and 
analog  data  is  transferred  at  10  times  per  second. 

Table  1,  C5-CPT  Cycle  Time  Required  for  I/O 
Transfers,  identifies  the  quantities  of  I/O 
channels  required  and  the  quantity  of  CPU  Instruc¬ 
tion  cycles  required.  These  figures  do  not  include 
the  set  up  time  required  by  the  CPU  to  initiate 
each  block  transfer.  A  microprocessor  would  save 
time  by  transferring  only  that  data  which  has 
changed  from  the  previous  value,  at  rates  con¬ 
sistent  with  the  particular  type  of  input  or  output. 


Table  1.  C5-CPT  Cycle  Time  Required  for 
I/O  Transfers 


I/O 

TYPE 

QUANTITY 

16-BIT 

CHANNELS 

MAX  I/O 
RATE 

MAX  DMA 

RATE 

CPU 

CYCLES 

PER  SEC 

TOTAL 

DO 

124 

3  us/chan. 

1  transfers 
ea.  3  cycles 

5 

620 

DI 

160 

3  ys/chan. 

2  transfers 
ea.  3  cycles 

5 

800 

CO 

344 

3  ys/chan. 

1  transfer 
ea.  3  cycles 

3440 

Cl 

64 

75  ys/chan. 

2  transfers 
ea.  3  cycles 

640 

Packing  and  Unpacking 


Another  task  which  is  directly  related  to  the 
quantity  of  data  words  being  transferred  is  the 
procedure  of  packing  and  unpacking  I/O,  and  applies 
to  digital  data.  This  procedure  involves  storing 
each  bit  of  a  data  word  as  a  complete  word  in 
memory  as  shown  in  Figure  1.  The  unpacked  format 
allows  the  DI's  and  DO's  to  be  in  a  form  more 
palatable  to  most  FORTRAN  based  programs.  This 
requirement  is  based  on  the  fact  that  most  minis 
do  not  have  good  bit  testing  capability. 

It  is  a  simple  task  to  check  the  sign  bit  of 
a  data  word  in  a  real  time  operating  program; 
therefore  it  takes  much  less  time  to  use  unpacked 
data  than  it  would  take  to  perform  bit  testing. 

This  frees  up  the  main  computer  program  from  bit 
testing  and  data  manipulation.  Extra  time  is  re¬ 
quired  to  perform  this  packing  and  unpacking  during 
I/O  transfers.  To  keep  this  time  at  a  minimum,  the 
CPU  compares  new  input  data  to  the  previous  data. 
This  requires  two  areas  in  memory,  one  to  store  the 
data  being  transferred  as  identified  under  polling, 
and  a  second  one  to  store  the  previous  data  for 
comparison  purposes.  If,  during  the  comparison 
process,  the  CPU  detects  a  difference  between  the 
old  and  the  new  data,  the  CPU  will  then  unpack  the 
input  data,  a  bit  at  a  time,  into  the  working 
memory.  Table  2  summarizes  the  Harris  Slash  5 
iterative  instructions  required  for  each  DI  word 
and  shows  the  total  computer  cycles  required  for 
the  C5A  trainer.  All  output  data  is  packed  into  a 
16-bit  word.  Table  3  is  a  summary  of  instruction 
cycles  required  to  pack  all  DO  words  for  the 
C5A-CPT.  The  number  of  total  cycles  required  is 


120 


Bit  # 


Bit  1 

— 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

Bit  2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

Bit  3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Bit  4 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

- - - - -  , 

1 

_ _ _ _l 

Bit  15 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

Bit  16 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

UNPACKED  I/O  DATA 


Hem  Add  N 
Mem  Add  N  +  1 
Mem  Add  N  +  2 
Mem  Add  N  +  3 

Mem  Add  N  +  14 
Mem  Add  N  +  15 


Figure  1.  Data  Formats  for  Packed  and  Unpacked  Data. 


Table  2.  Instruction  Cycles  Required  to 
Compare  Input  Data 


INSTRUCTION 

NUMBER  OF  CYCLES 

TME  Transfer  memory 
to  E  Reg. 

2  Load  01  word 

CME  Compare  memory 

2  Compare  last  input  to 

to  E  Reg. 

present  input 

B02  Branch  on  0 

1  Branch  if  not  different 

AOJ  Increment 
buffer  address 

1 

AO I  Increment 
memory  address 

1 

AOK  Load  reg. 

1 

CMJ  Compare  for 
last  DI  word 

2 

BON  Branch  on  non 

1 

total 

11  cycles  x  160  DI  words 
=  1760  cycles 

X  5  per  second 

8800  cycles/sec. 

Table  3.  Instruction  Cycles  Required  to 
Pack  Output  Data 


INSTRUCTION 

NUMBER  OF  CYCLES 

TNJ  Load  #  bits  (16) 

1  16  bits  per  word 

12A  Transfer  zero 
to  A 

1  Zero  A  register 

TME  Transfer  memory 
to  E  reg. 

3  Load  data  from  memory 

'X16 

LRD  Left  rotate 
double 

2  Shifts  sign  bit  into  A 
Register 

AO I  Increment  to  next 
memory 

1  Add  1 

BWJ  Return  for  next 
bit 

1  Do  all  16  bits 

TAM  Transfer  word  to 
memory 

> 

2  Transfer  work  to  RTI 
buffer 

AOK  Increment  K 

1 

■  XI 

CMK  All  words  done? 

2 

BNK  Branch  if  not 
zero  to  top 

1 

((9x16)  +  6)  124  DO  words 
*  18600  cycles 

X  5  per  sec. 

_ 

93000  cycles/ sec. 

cc-.'  >**&&&%&**»* ,  •  ----<£g- 
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1760  for  a  DI  compare  and  18600  for  DO  packing,  for 
each  Iteration.  This  requires  over  100K  Instruc¬ 
tion  cycles  per  second.  Faster  Iteration  rates 
obviously  require  more  time. 

Since  digital  data  normally  represents  manual 
functions,  significant  CPU  time  can  be  saved  by 
transferring  only  unpacked  data.  At  first  glance 
this  would  seem  to  require  a  significantly  greater 
1/0  time  for  polling  purposes.  However,  there  are 
two  Important  factors  to  be  considered:  first, 

I/O  time  takes  less  time  than  comparing  and  unpack¬ 
ing,  and  second,  the  microprocessor  can  be  pro¬ 
gramed  to  transfer  only  changed  data  as  follows. 

The  transfer  of  data  which  contains  no  new 
Information  is  a  loss  of  valuable  time.  A  smart 
I/O  would  transfer  information  only  when  there  is 
new  data.  Because  of  the  continuous  nature  of 
analog  data,  the  arguments  for  transmitting  only 
changed  data  would  be  applicable  primarily  to 
digital  data. 

This  task,  combined  with  the  capability  of 
transmitting  unpacked  data,  would  result  In  minimal 
processor  cycles  each  second.  Transmitting  of 
unpacked  data  could  be  accomplished  In  either  of 
two  ways;  a  block  of  data  preceded  by  a  single 
address  or  an  address  transmitted  for  each  data 
word.  The  Harris  External  Block  Controller  (XBC) 
provides  a  means  for  automatic  data  transfer 
between  memory  and  a  selected  peripheral  device  on 
a  random  access,  cycle-stealing  basis.  This  device 
would  allow  transfers  directly  with  the  memory 
location  used  by  the  operating  program,  saving 
time  and  memory.  Table  4  shows  the  I/O  buffer 
space  savings  because  of  the  elimination  of  inter¬ 
mediate  storage  requirements  and  the  number  of 
CPU  cycles  saved  by  elimination  of  packing  and  un¬ 
packing  requirements.  State  change  detection  is 


an  Itegral  part  of  data  transfer  reduction. 
American  Micro  Devices  has  announced  a  new  inte 
grated  circuit,  P/N  AM29837,  which  is  called  a 
Bit  Mapped  I/O  Port  and  Is  shown  in  Figure  2. 


Table  4.  Benefits  of  Eliminating 
Packing  and  Unpacking 


I/O  BUFFER  SPACE  SAVINGS 

CPU  CYCLES  SAVED 

DO  =  124  x  2  =  248 

Polling  time 

5500  cycles/ 

sec 

DI  =  160  x  2  =  320 

Input  un- 

8800  cycles/ 

packing 

sec 

568  words 

Output  pack- 

93000  cycles/ 

ing 

sec 

107300  cycles/^ 

sec  | 

This  part  would  simplify  the  microprocessor 
tasks  of  determining  if  there  are  changes  on  Inputs 
by  providing  information  regarding  changes  without 
the  overhead  of  comparing  to  previous  data  in  the 
processor. 

Scaling  and  Curve  Fitting 

Scaling  and  curve  fitting  are  additional 
tasks  which  may  be  relegated  to  the  microprocessor 
peripheral.  These  tasks  relate  to  the  conversion 
between  normalized  values  for  equation  solving 
purposes  In  the  computer  and  engineering  values 
required  for  the  I/O  system.  Each  analog  Input  in 
the  C5-CPT  trainer  goes  through  the  following 
process  and  requires  the  Indicated  number  of 
computer  cycles: 


Figure  2.  Bit-Mapped  I/O  Port 
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Number  of  Cycles 

TMA  Load  accumulator  2 

with  analog  Input 

LLA  Left  shift  to  left  3 

justify  for  sign  bit 

RRA  Right  shift  to  right  3 

justify  with  sign  bit 

MYM  Multiply  to  a  £ 

normalized  value  of  ±  1 

14  cycles 

Times  quantity  of  x  64 

analog  inputs 

896  processor  cycles 
x  10  per  second 
8960  cycles/ sec. 

Each  analog  output  requires  a  conversion  from 
a  normalized  value  to  an  engineering  value.  For 
the  simplest  of  analog  outputs  this  requires  solv¬ 
ing  the  following  equation: 

y  =  mx+b 

where: 

y  =  engineering  value 
m  =  slope  of  the  line 
x  =  normalized  value 
b  =  y  intercept 

The  following  conversion  routine  is  typical 
for  solving  the  previous  equation: 

Number  of  Cycles 

Load  accumulator  with  value  2 

of  "x" 

Multiply  by  the  value  6 

of  "m" 

Add  the  value  of  "b"  2 

Shift  to  12-bit  format  3_ 

13 

Times  the  quantity  of  x  344 

analog  outputs  - 

4472  cycles 

x  10  per  second 

44720  cycles/sec. 

As  mentioned  earlier,  this  represents  the 
simples'-  conversion  of  analog  outputs  and  is  re¬ 
quired  for  each  output.  Some  cockpit  flight 
instruments,  because  of  nonlinearities,  require 
several  straight  line  segments  to  cover  the  range 
of  engineering  values.  This  requires  additional 
checking  routines  to  first  determine  which  segment 
of  the  curve  the  value  falls  into  and  also  requires 


additional  memory  to  store  the  associated  con¬ 
stants.  For  a  single  straight  line  curve,  two 
constants  must  be  stored;  the  values  of  m  and  b. 

Instruments  which  require  dual  continuous 
outputs,  such  as  compasses  and  two  speed  devices 
including  altimeters  and  attitude  direction  indi¬ 
cators,  require  additional  handling  to  convert  the 
output  data  Into  sine  and  cosine  format.  To 
calculate  this  value  Involves  solving  a  polynomial 
equation  In  the  following  form: 

y  =  CjX3  +  C2x2  +  C3x  +  C4 

Typical  conversion  time  for  an  equation  of 
this  form  requires  200  us  for  each  conversion  in 
the  Harris  Computer.  In  the  C5-CPT  trainer  there 
are  (23x2)  sine/cosine  conversion  routines  which 
require: 

46x200  =  9.2  ms  (9.2K  Instruction  cycles) 
x  10  per  sec.  =  92000  cycles/sec. 

Summary  of  1/0  Tasks 

The  previous  tasks  associated  with  I/O  han¬ 
dling  require  an  appreciable  amount  of  time.  For 
the  basic  tasks  identified,  total  CPU  time  is 
summarized  as  follows: 

Polling  5500  cycles/sec. 

Packing  and  unpacking  101800  cycles/sec. 

Analog  inputs  8960  cycles/sec. 

Curve  fitting  44720  cycles/sec. 

Sine/cosine  conversion  92000  cycles/sec. 

252980  cycles/sec. 

At  1  us  per  instruction  cycle  this  represents 
25.3%  of  each  second  of  processing  time.  It 
should  be  pointed  out  that  the  tasks  defined  here 
do  not  include  overhead  of  set-up  time  required 
nor  do  they  include  the  additional  time  for  those 
analog  outputs  which  must  be  tested  for  multi¬ 
segment  curve  fitting.  Also  not  included  is  the 
actual  time  required  to  unpack  a  changed  DI  input. 
Another  factor  which  increases  the  total  time  is 
that  all  channels  are  not  contiguous  and  are  split 
between  four  separate  devices. 

The  previous  examples  of  tasks  which  could  be 
off-loaded  to  a  peripheral  processor  may  not  seem 
too  significant  once  a  computer,  such  as  the 
Harris,  is  selected  for  a  specific  trainer,  such 
as  the  C-5  CPT.  In  the  beginning  there  is  always 
plenty  of  spare  memory  and  time  available  to  meet 
specification  requirements.  However,  because  of 
additional  requirements  and  ECP  add-on,  this  partic¬ 
ular  trainer  ran  out  of  memory  space  and  required  a 
second  Slash  5  minicomputer  to  handle  all  the  pro¬ 
cessing  tasks.  Once  the  original  design  used  the 
minicomputer  as  the  sole  processing  device,  the 
best  solution  to  add-on  requirements  was  another 
mini  in  order  to  minimize  system  design  changes. 

MICROPROCESSORS  IN  RADAR  SIMULATION 

Microprocessors  are  increasingly  taking  on  the 
tasks  once  performed  only  by  minicomputers  in  the 
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area  of  radar  simulation.  Many  of  the  more  recent 
16/32-b 1 t  microprocessors  equal  or  exceed  presenc 
day  minicomputer  capabilities  In  speed  and  calcula¬ 
tion  precision,  at  less  cost.  However,  the 
approach  to  a  solution  using  microprocessors  may 
not  be  simply  supplanting  the  minicomputer. 

Concepts  of  parallel  vector  and  multiprocessing 
are  much  more  easily  and  cheaply  realized  using 
microprocessors  than  using  minicomputers. 

The  design  approach  for  radar  simulation  sys¬ 
tems  using  microprocessors  must  be  optimized  to 
achieve  greatest  cost/performance  yield.  Each 
hardware  and  software  subsystem  must  be  evaluated 
for  the  individual  application  as  well  as  the 
future  applications  to  Increase  the  hardware/soft¬ 
ware  life  cycle  of  the  final  product.  Hardware 
tradeoffs  such  as  multiprocessor/shared  resource 
designs,  as  well  as  off-the-shelf  approaches  to 
specific  subsystems,  must  be  evaluated.  Software 
tradeoffs,  such  as  selection  of  important  Instruc¬ 
tion  set  features  as  well  as  selecting  the  most 
efficient  software  language  mix,  must  be  contem¬ 
plated. 

Simulation  of  a  radar  system  containing  large 
numbers  of  targets  or  multiple  gaming  areas  re¬ 
quires  multiprocessing  and  shared  resources.  This 
simulation  requires  multiple  semaphore  switching 
of  shared  memory  and  peripherals.  Semaphore 
switching  may  be  accomplished  directly  by  the 
microprocessor  itself,  or  indirectly  via  an  intel¬ 
ligent  resource  controller.  The  advantage  of  a 
resource  controller  is  that  a  processor  can  make 
a  request  for  a  shared  resource,  then  continue 
processing.  The  resource  controller  would  continue 
to  poll  the  status  of  the  resource  until  it  is 
free  for  use,  then  hold  possession  of  the  resource 
while  signaling  the  requesting  processor.  This 
saves  processor  time  and  allows  efficient  use  of 
system  resources. 

Hardware  subsystems  producing  clutter,  jamming 
and  weather  effects  must  be  evaluated  for  general 
design  approach.  Software  generation  of  these 
effects  is  quite  time  consuming.  It  is  more 
prudent  to  free  up  the  target  generator  processor 
by  dedicating  a  separate  microcomputer  to  the  task 
of  generating  these  effects.  Since  setup  selection 
(display )t  or  de-selection  (blanking)  of  these 
phenomena'  occurs  infrequently  during  the  course  of 
simulation,  a  'slow'  serial  RS-23Z  link  to  the 
target  processor  may  be  used.  This  link  makes  the 
clutter/weather  generator  usable  with  any  proces¬ 
sor.  The  target  processor  would  down-load  set-up 
information  such  as  type,  intensity,  wind  boundary, 
speed,  etc.  to  the  clutter/weather  processor.  The 
peripheral  processor  would  then  execute  the  ap¬ 
propriate  algorithms  and  output  the  raw  video  via 
a  video  generator  port.  This  port  would  be  de¬ 
signed  to  accommodate  expected  variations  in 
sweep  speed,  video  level,  etc.  The  goal  of  this 
design  approach  is  to  have  off-the-shelf  hardware/ 
software  for  clutter/weather  generation,  indepen¬ 
dent  of  target  processor  type,  with  a  savings  of 
target  processor  time. 

Radar  simulation  requires  a  specific  set  of 
demands  on  computing  systems.  Since  all  calcula¬ 
tions  of  targets  and  radar  clutter/jamming  must  be 
done  in  real  time,  a  fast  instruction  set  is 
essential.  Key  areas  of  performance  within  the 


instruction  set  are  fast  multlply-divide  with 
number  crunching  of  at  least  32  Bits,  context 
switching  during  interrupt  vectoring,  looping 
primitives,  and  flexible  addressing  modes.  In 
addition,  instruction  execution  speed  is  greatly 
enhanced  with  a  large  directly  addressable  memory 
capability  due  to  the  minimization  of  memory 
management  overhead. 

Radar  simulation  also  requires  an  efficient 
mix  of  software  languages  to  achieve  the  required 
performance  level  for  the  specific  system.  High 
performance  systems  require  calculations  for  tar¬ 
get  dynamics,  radar  sorting,  radar  conditioning 
effects,  and  routines  for  I/O  handling  to  be 
written  in  assembly  language.  In  addition,  the 
executive  should  be  written  in  assembly  language 
to  quickly  assign  new  tabs.  Utilization  of  as¬ 
sembly  language  allows  software  functional  entities 
that  are  time  constrained  and  iterative  to  be 
processed  at  the  highest  speed  of  the  microproc¬ 
essor.  Use  of  assembly  language  has  the  added 
advantage  of  being  compact,  thus  saving  memory  and 
recurring  hardware  costs. 

Software  functions  that  are  not  time  con¬ 
strained  may  be  more  quickly  written  using  higher 
level  languages,  which  are  also  more  'portable'. 

Low  priority  functions,  including  data  entry/dis¬ 
play  and  scenario  generation,  should  be  written  in 
higher  level  languages  such  as  Pascal,  FORTRAN  or 
ADA.  Portable  programs  may  be  written  for  general 
use  as  off-the-shelf  software,  thus  reducing  costs 
and  time  for  each  new  simulator. 

Low  performance  systems  should  have  the  entire 
radar  model  and  control  functions  written  in  a 
higher  level  language.  The  I/O  handler  would  be 
the  only  function  written  in  assembly  language. 

The  advantage  of  this  approach  is  "portability"  of 
software  that  is  more  likely  to  be  off-the-shelf. 
The  disadvantages  are  greater  memory  consumption, 
higher  recurring  hardware  costs,  and  a  lower 
number  of  targets  that  may  be  handled  by  an  in¬ 
dividual  processor. 

The  following  paragraphs  discuss  significant 
features  of  microprocessors  as  applied  to  radar 
trainers  and  summarize  the  distinct  advantages  of 
the  microprocessor. 

Microprocessor  Applications  in  Radar  Trainers 

Tables  5  and  6  compare  minicomputer  and 
microprocessor  based  training  devices. 

Table  5,  CPU  Internal  Structure,  summarizes 
the  basic  structure  of  the  CPU  devices  used  on 
various  radar  training  devices.  These  devices 
range  from  the  Harris  Slash  5  minicomputer  used  in 
1971-2  on  the  Forward  Area  Alert  Radar  trainer  for 
the  Army  and  the  15G19  radar  trainer  for  the  Navy 
to  the  present  Motorola  68000  microprocessor  being 
designed  for  the  AN/GPN-T4(V)  Air  Force  trainer. 

An  earlier  T4  device  delivered  in  the  late 
70's  to  the  Air  Force  used  an  IMP16-L  microproc¬ 
essor  as  did  the  15G20  which  has  been  delivered  to 
the  Navy  and  the  Marines.  Table  6,  CPU  Development 
Facilities,  shows  the  features  available  with  each 
of  the  CPU  systems  for  the  radar  trainers  refer¬ 
enced  above. 
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Table  5.  CPU  Internal  Structure 


CPU 

HARRIS  6024/5(MlNl) 

IMP16-L (MICRO) 

68000  (MICRO) 

FAAR,  ARMY 

AN/GPN-T4,  AIR  FORCE 

DEVICE 

15G19  NAVY 

15G20,  NAVY,  MARINES 

AN/GPN-T4(V),  AIR  FORCE 

Byte-word  (24  bit)  long  word 

16  bit  word,  limited  byte  and 
long  word  operations 

Byte,  word,  and  long  word 
instructions  with  little 
restriction 

6  registers  (24  bit) 

2  working  registers 

8  data  registers  (32  bits) 

2  address  registers 

7  address  registers  (32  bits) 

No  stack 

16  word  internal  stack 

Stack  in  external  memory 
without  restriction 

65K  address  space 

65K  address  space 

8M  direct  address  space 

Vectored  interrupts  only 

Nonvectored  interrupts  only 

Vectored  or  autovectored 
interrupts  or  both 

Data  transfers  controlled  by 
Harris  data  channel;  no  user 
connection  to  bus 

Time  shared  address  and  data 

1  ines 

Separate  address  and  data 
lines 

Table  6.  CPU  Development  Facilities 


1971-72  MINICOMPUTER 

HARRIS  6024/5 

1974  MICROPROCESSOR 

NATIONAL  SEMICONDUCTOR 

IMP-162 

1980  MICROPROCESSOR 

MOTOROLA  68000 

Programer1 s  control  panel,  no  CRT 

Programer's  control  panel,  no  CRT 

CRT-Keyboard  terminal 

Punch  cards  or  paper  tape 

Punch  cards  or  paper  tape 

Floppy  disk  handlers 

Own  assembly  language  plus 

FORTRAN  IV 

Own  assembly  language  only 
(FORTRAN  required  external 
processor) 

Own  assembly  language, 

FORTRAN  77,  Pascal 

Assembler  and  editor 

'Conversational'  assembler  as 
limited  editor 

Assembler  and  editor  (word 
processing,  CRT  oriented) 

Debugger  but  no  incircuit 
emulator 

Debugger  but  no  incircuit 
emulator 

Debugger  plus  incircuit 
emulator 

No  memory  space  protection 

No  memory  space  protection 

Optional  memory  space 
protection 

User  mode  only 

User  mode  only 

Supervisory  and  user  modes, 
privileged  and  nonprivileged 
instructions 

Computers  in  Radar  Simulation  in  1973  and  1980 

A  $30,000  minicomputer,  standing  as  high  as 
the  programer,  seemed  like  a  reasonable  solution 
to  a  real  time  radar  simulation  development  in 
1973.  This  punch  card  based  system  easily  computed 
dynamics  of  16  targets  at  25  discrete  ranges, 
although  system  integration  with  our  hardware  was 


sometimes  baffling  and  usually  very  slow.  There 
were  no  logic  analyzers  then.  In  addition,  the 
required  card  reader,  tape  handler,  chain  printer, 
1/0  channel  boards,  cables,  other  hardware,  and 
software  support  pushed  the  total  development 
package  t  ■  nearly  triple  that  of  the  computer 
alone,  yet  only  the  computer  and  a  paper  tape 
reader  needed  to  be  delivered  with  the  simulator. 
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In  1980,  the  same  $30,000  bought  a  16-bit 
microprocessor  development  system  arid  a  software 
package  for  another  radar  simulation.  The  system 
Included  a  processor,  a  keyboard-CRT  terminal,  a 
printer,  a  dual  floppy  disk  handler,  and  an  In- 
circuit  emulator  pod.  The  software  package 
Included  a  word  processing-oriented  editor  and  a 
debugger.  No  more  keypunch  machine  or  punch  card 
files.  The  new  printer  was  slower  than  the  old 
chain  printer  but  was  the  size  of  a  typewriter 
Instead  of  a  small  refrigerator. 

The  job  for  which  this  mlcropi ocessor  system 
was  bought  required  delivery  of  only  a  processor, 
some  memory,  and  a  DMA  controller.  Each  was  to 
be  put  on  a  9  x  10-Inch  printed  circuit  board. 

Each  delivered  radar  simulation  system  then  in¬ 
cluded  only  about  $3,000  worth  of  computation 
hardware,  about  one-tenth  of  that  required  in  1973. 
Since  the  program  and  all  diagnostics  were  resident 
In  ROM  on  the  memory  board,  no  other  peripherals 
were  needed  in  the  delivered  system. 

Hardware  integration  wasn't  so  baffling.  Now 
we  could  plug  the  development  system  into  the 
socket  where  our  microprocessor  chip  was  to  go, 
rather  than  connect  a  mini  via  a  peripheral  channel 
as  before.  Single  step  execution,  break  points, 
register  examination,  traps,  and  data  manipulation 
all  took  place  at  a  terminal  with  the  flexibility 
of  a  keyboard  and  CRT  display,  rather  than  at  a 
control  panel  with  switches  and  lamps. 

Of  course,  minicomputer  development  hasn't 
stood  still  since  1973.  But  for  us,  the  contrast 
in  development  of  a  computer-based  system  for 
radar  simulation  between  1973  and  1980  reflects 
work  in  two  different  worlds. 


MICROPROCESSOR  PERFORMANCE 

The  performance  of  the  microprocessor,  includ¬ 
ing  speed,  instruction  set,  I/O  rate  and  software, 
is  an  important  faccor  when  considering  offloading 
tasks  from  a  minicomputer.  If  a  task  becomes  more 
difficult  or  time  consuming  to  handle  in  the  micro, 
then  very  little  has  been  gained  unless  production 
savings  offset  development  costs.  Benchmarking 
provides  a  means  for  evaluating  a  processor's 
performance  of  certain  tasks.  A  group  at  Carnegie 
Mellon  University  compiled  a  set  of  programs  in 
1976  for  benchmarking  minicomputers . (Z)  EDN 
magazine(3)  published  the  results  of  a  comprehen¬ 
sive  benchmark  study  of  four  major  16-bit 
microprocessors:  the  Digital  Equipment  Corp  LSI 
11/23,  Intel  8086,  Motorola  68000  and  Zilog  Z8000. 
EDN's  tests,  while  being  a  subset  of  the  Carnegie- 
Mellon  set,  specifically  exclude  benchmarks 
dealing  with  floating  point  math  or  virtual- 
memory  handling  because  most  16-bit  microprocessors 
do  not  support  floating  point  operations  or  virtual 
memory.  Also  excluded  were  two  benchmarks  that 
require  extensive  number  crunching  capability 
(Fourier  transforms  and  Runga-Kutta  integration). 
The  results  of  the  seven  remaining  microprocessor 
benchmarks  are  shown  in  Table  7. 


Table  7.  Microprocessor  Benchmark  Execution  Times 


EXECUTION  TIME  IN  MILLISECONDS 

BENCHMARK 

LSI  11/23 

8086 

68000 

Z8000 

A.  1/0  Interrupt 
Kernel 

114 

126 

33 

42 

B.  I/O  Kernel  with 
FIFO  Processing 

1196 

348 

390 

436 

E.  Character  String 
Search 

996 

193 

244 

237 

F.  Bit  Set,  Reset, 
Test 

799 

122 

70 

123 

H.  Linked-List 
Insertion 

592 

- 

153 

237 

I.  Quicksort 

- 

115,669 

33,527 

115,500 

K.  Bit-Matrix 
Transposition 

1517 

820 

368 

646 

The  following  clock  speeds  were  used. 


LSI  -  11/23 

3.33  MHZ 

8086 

10.00  MHZ 

68000 

10.00  MHZ 

Z8000 

6.00  MHZ 

Table  8  tabularizes  the  benchmark  code  bytes 
required  for  the  same  microprocessors  and  compares 
them  to  the  Interdata  8/32  mincomputer  which  was 
the  superior  device  of  the  Carnegie  Mellon  tests. 

Table  8.  Benchmark  Code  Bytes 


CODE  BYTES 

INTER¬ 

DATA 

BENCHMARK 

LSI  11/23 

8086 

68000 

Z8000 

8/32 

A.  I/O  Interrupt 
Kernel 

20 

55 

24 

18 

26 

B.  I/O  Kernel  with 
FIFO  Processing 

86 

85 

118 

106 

98 

E.  Character  Strinc 
Search 

76 

70 

44 

66 

120 

F.  Bit  Set,  Reset, 
Test 

70 

46 

36 

44 

82 

H.  Linked-List 
Insertion 

138 

94 

106 

96 

148 

I.  Quicksort 

- 

347 

266 

386 

426 

K.  Bit-Matrix 
Transposition 

152 

88 

74 

110 

130 

126 


From  Table  7  and  8  we  note  that  all  micro¬ 
processors  are  not  equal  In  performance.  It  Is 
therefore  up  to  the  Individual  user  to  compare  his 
particular  requirements  against  the  various  micros 
and  perform  a  trade  off  evaluation.  It  should  be 
noted  that  the  fastest  clock  speed  Is  not  neces¬ 
sarily  the  device  with  the  best  performance.  For 
Instance,  the  Z8000  at  a  clock  speed  of  6.00  MHz 
performs  benchmark  K  (bit-matrix  transposition) 
faster  than  the  68000  with  a  clock  speed  of  10.0 
MHz;  however.  It  requires  more  bytes  of  code. 

From  Table  8  we  note  a  significant  savings  In  the 
number  of  code  bytes  required  for  most  of  the 
microprocessor  benchmarks  as  compared  to  the 
Interdata  8/32  minicomputer. 

CONCLUSIONS 

It  has  been  shown  how  the  microprocessors  can 
be  used  to  enhance  the  performance  of  a  training 
simulator.  Specific  tasks  can  now  be  performed  on 
the  microprocessor  at  much  greater  speeds  and 
economy  than  on  the  minicomputer;  for  example, 
routine  Iterative  tasks  of  data  handling,  number 
crunching,  hardware  monitoring  and  radar  target 
generators. 

With  the  current  technological  development  of 
microprocessors,  we  can  expect  these  benefits  to 


Increase  and  that  more  and  more  tasks  may  be  off¬ 
loaded  onto  the  microprocessor.  It  Is  thus 
Incumbent  upon  the  simulator  systems  designers  to 
economically  Incorporate  microprocessors  Into  their 
particular  application. 
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ABSTRACT 


Computer 
provide 
traininq 
sof tware 


voice  response  and  computer  speech  recognition  can 
a  valuable  supplementary  traininq  aid  for  military 
systems.  For  traininq  devices  based  on  media  or 


which  is  hard  to  chanqe,  voice  can  provide  supplementary 
information  and  allow  spoken  amendments  to  course  information. 
For  simulators  which  attempt  physical  fidelity  to  a  particular 
speech  provides  a  medium  which  can  communicate 
and  from  the  student  without  interfering  with  the 


defense  system, 
information  to 


defense  system  displays  or  controls. 


It  is  feasible  to  add  computer  voice  response  o'  telephone 
quality  to  most  traininq  systems.  Such  voice  response  can  be 
entered  and  changed  by  simply  speaking  into  a  microphone.  At 
least  thirty  minutes  of  such  speech  can  be  stored  and  retrieved 
digitally  in  a  simple  hardware  implementation. 


Speech  recognition  capability  can  add  a  further  dimension  to  a 
voice  training  aid,  allowing  the  trainee  to  make  requests  or  to 
answer  multiple-choice  questions. 


INTRODUCTION 

In  this  paper,  we  will  look  at.  two  major 
cateqories  of  traininq  situations  which 
vterisei  1)  with  simulators  or  part-task 
trainers,  and  2)  with  traininq  devices, 
such  as  videodisc.  or  computer-aided 
instruction  systems.  This  paper  will 
discuss  where  we  believe  that 
computer-voice  response,  i.e.,  the  ability 
of  the  computer  to  communicate  with  the 
student  by  voice,  can  be  an  important 
supplement  to  these  traininq  technologies, 
making  these  technologies  more  effective. 

will  also  discuss  how 
speech-recognition  capabilities,  i.e.,  the 
ability  of  the  student  to  respond  to  the 
computer  by  voice,  can  be  a  valuable 
supplement  to  computer  voice  response  in 
some  situations.  We  will  discuss  how 
current  state-of-the-art  hardware  and 
software  can  be  used  to  meet  the 
requirements  generated  by  these 
appl icati ons. 

There  have  been  more  specialized 
discussions  elsewhere  of  the  role  of 
speech  technology  in  training, 
particularly  in  situations  where  the 
response  to  be  trained  is  a  vocal  response 
and  in  which  the  specific  syntax  and  words 
used  are  part  of  the  traininq  (1,3-3). 

SIMULATORS  AND  PART-TASK  TRAINERS 

There  is  a  wide  range  of  traininq  systems 
being  developed  for  military  and  other 
applications.  These  devices  include  very 
complex  weapon  system  trainers.  Weapon 
system  trainers  simulate  in  real  time  the 
response  of  the  operational  weapon  system 
to  the  trainee's  actions.  At  the  other  end 
of  the  spectrum  are  more  simple 


familiarization  trainers  where  the  device 
responds  to  a  limited  number  of  console 
actions  to  give  the  trainee  experience 
with  a  particular  sot  of  switches  and 
contr  o).  s. 

Simuljtprs  and  Computer-Aided  Instruction 

The  major  characteristic  of  such  trainers 
is  that  they  attempt  to  simulate  to 
varyinq  degrees  the  actual  system  for 
which  the  training  is  beinq  undertaken. 
Thus,  the  displays,  switches,  and  other 
means  of  communication  between  the  trainee 
and  the  system  are  those  present  in  the 
actual  weapon  system.  If  we  wish  to 
automate  such  a  system  to  provide 
computer-aided  instruction  as  a  supplement 
to  the  instructor,  we  are  faced  with  the 
difficulty  that  we  do  not  wish  to 
interfere  with  the  fidelity  of  the 
student -system  interaction.  We  do  not 
wish  for  there  to  appear  on  the  CRT  screen 
of  the  weapon  system  a  question  or  a 
prompt  that  would  not  be  available  in  the 
actual  weapon  system.  Misuse  of  a 
si mul ator /trai ner  as  a  computer-aided 
instruction  device  could  too  easily  result 
in  confusing  the  trainee  between  what  he 
can  expect  in  an  operational  situation  and 
the  traininq  situation.  We  would 
therefore  like  to  have  such  machine-based 
instruction  occur  without  interference 
with  the  operation  of  the  traininq  system. 

Having  the  computer — based  instruction  be 
delivered  by  spoken  messages  is  probably 
the  ideal  alternative  for  this  type  of 
system,  particularly  since  the  student  is 
used  to  receiving  vocal  instructione  from 
the  instructor  while  he  is  operating  the 
training  system. 
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Let's  consider  a  more  specific  example. 
Suppose  Me  wished  to  teach  emergency 
procedures  with  a  part-task  trainer.  The 
system  might  be  such  that,  when  a 
particular  weapon  system  panel  failed,  the 
procedure  was  to  use  keyboard  entry. 
Because  the  major  part  of  the  training 
will  be  for  normal  operation,  training  in 
the  emergency  procedures  can  be  a  poor  use 
of  time  for  an  instructor,  particularly  if 
these  procedures  are  complex  enough  to 
require  many  repetitions  for 

familiarization.  Yet,  no  one  would  argue 
aqainst  the  importance  of  training  in 
these  procedures. 

An  example  of  the  interaction  with  a 
speakinq  computer-ai ded-  instruction  (CAD 
system  miqht  be  as  follows.  The  CA1 
system  miqht  say,  "Suppose  the  firing 
panel  has  failed.  What  would  you  do  in 
order  to  initialize  your  missile  battery 
for  firing?"  The  user  responds  as  he 
would  in  an  operational  situation,  typing 
in  his  response.  If  the  system  is 
implemented  such  that  the  CAI  system  is 
aware  of  what  response  was  typed  in,  the 
machine  can  respond  to  a  correct  answer  by 
sayinq,  "That  is  correct."  and  proceeding 
to  the  next  step  in  the  instruction.  If 
tne  answer  is  incorrect,  the  CAI  system 
mighi  indicate  the  nature  of  the  error; 
for  example,  it  miqht  say  "You  transposed 
a  C  and  an  A  in  the  command.  Please  try 
aqain,"  or  "You  used  the  command 
appropriate  for  a  failure  of  the 
naviqatiun  panel  instead  of  the  firing 
panel,  please  try  aqain." 

If  the  CAI  system  is  not  set  up  in  such  a 
way  that  it  can  read  the  specific  response 
of  the  trainee,  it  may  simply  state  the 
correct  response  and  ask  him  if  he 
requires  further  explanation.  This 

approach  requires  that  the  student  be  able 
to  interact  directly  with  the  CAI  system 
to  indicate  that  he  or  she  needs  more  help 
or  a  repetition  of  the  instructions.  For 
the  reasons  previously  discussed,  it  would 
be  useful  if  the  student's  responses  could 
be  vocal.  It  would  be  useful  if  he  or  she 

could  state  what  the  response  was  and  have 

the  computer  comment  upon  it,  as  if  the 

CAI  system  were  aware  of  the  keys  pressed. 

Even  in  a  situation  where  the  CAI  system 
is  aware  of  the  key  presses,  there  should 
be  a  mechanism  for  the  student  to 
communicate  with  the  system  to  request 
help,  a  hint,  or  more  detailed 

expl anation. 

Speech  Technology  Requirements 

Let  us  discuss  the  voice  response  and 

recognition  technology  which  would  be 
desirable  for  voice-aided  training  as  we 
have  described  it. 


Voice  response.  A  hiqhly  desirable 
requirement  is  that  the  speech  be  easily 
intelliqible  and  natural  speech.  The 
student  is  learninq  a  difficult  task  and 
it  is  inappropriate  for  him  to  also  be 


learning  to  understand  unnatural  speech. 
We  would  like  the  speech  to  be  a*  if  it 
were  coming  from  a  recording. 

It  would  also  be  useful  if  the  speech 
response  system  could  produce  sounds  as 
well;  or  example,  it  might  be  useful  to 
explain  to  the  student  that  when  he  had 
pressed  a  particular  sequence  of  buttons, 
he  would  hear  a  particular  sound  and  have 
the  system  replicate  that  sound. 

Similarly,  having  different  voices  would 
be  useful;  for  example,  one  could  use  a 
female  voice  for  instructions  about  the 
CAI  procedure  and  a  male  voice  for  the 
instructional  material. 

Another  requirement  is  that  the  vocabulary 
of  the  speech  response  system  be 
essentially  unlimited  to  allow  instruction 
to  be  designed  and  changed  without 
artificial  constraints. 

A  third  requirement  is  that  there  be  no 
significant  delay  in  the  beqinninq  of  a 
vocal  response,  even  if  the  content  of 
that  response  depends  upon  the  student's 
actions.  If  there  is  a  differing  response 
for  a  correct  answer  than  an  incorrect 
answer,  the  machine  must  immediately 
present  the  appropriate  response.  This  is 
also  the  case  if  the  stud  ”'t  requests  a 
review,  more  detajfcl  ,  or  <  hint. 

We  must  also  require  that  there  be  a 
capability  for  storing  sufficient  speech 
to  carry  the  whole  traininq  session.  This 
would  probably  require  that  a  minimum  of 
fifteen  minutes  of  speech  be  stored  in  the 
speech  response  system.  It  is  unlikely 
that  more  than  an  hour  of  speech  would  be 
required  for  any  sinqle  traininq  session, 
considering  time  allowed  for  student 
response  and  for  repetition  of  material. 
A  subsidiary  requirement  is  that  the 
speech/material  be  changeable  by  reloading 
the  system  in  some  way  from  some  off-line 
storage  medium,  such  as  tape,  so  that  the 
course  content  could  be  easily  changed  to 
another  hour  of  speech. 

A  hiqhly  desirable  capability  is  that  the 
system  allow  the  speech  content  of  the 
course  material  to  be  easily  changed. 
Because  speech  is  such  a  natural  medium  in 
which  to  teach,  the  full  advantage  of  the 
speech  response  technology  will  not  be 
realized  unless  the  instructor  or  course 
designer  can  readily  chanqe  portions  of 
the  spoken  material.  This  can  be  required 
for  a  number  of  valid  reasons; 

1)  The  instructor  may  discover  that  the 
students  have  difficulty  with  a 
certain  portion  of  the  course  and 
require  additional  instructions  that 
are  not  in  the  present  course 
mater i al . 

2)  The  weapon  system  itself  may  chanqe, 
requiring  that  the  course  material  be 
chanqed  to  reflect  this.  (It  is 
particularly  advantageous  when  the 
course  material  can  compensate  for  an 
incorrect  response  of  the  weapon 
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system  trainer,  due  to  a  lata  change 
In  the  operational  weapon  eystem. > 

3)  An  instructor  may  discover  that  a 
certain  portion  of  the  instructional 
material  is  wrong  or  confusing. 

It  would  be  ideal  if  the  instructor  could 
change  the  course  material  simply  by 
indicating  the  portion  of  the  course  to  be 
changed  and  then  simply  dictate  vocally 
the  change  or  addition. 

A  final  requirement  is  that  the  addition 
of  the  voice  response  have  a  minimum 
impact,  if  any,  on  the  hardware  and 
software  of  the  training  system.  Given 
the  difficulty  of  developing  training 
systems,  any  significant  increase  in  the 
complexity  added  to  that  task  by  a  CAI 
adjunct  would  be  a  severe  impediment  to 
the  addition  of  such  capability.  For  this 
reason,  it  is  probably  inadvisable  to 
integrate  the  voice-response  capability 
into  the  weapon-system  hardware  and 
software. 

It  is  modern  design  philosophy  for 
computer-based  systems  to  modularize  as 
much  as  possible  to  simplify  the  software 
development.  The  implications  for  the 
computer — voice  response  in  this  context  is 
that  it  would  be  appropriate  for  the 
device  to  be  a  stand-alone  device  where 
changes  in  the  spoken  material  were  made 
independently.  The  communication  with  the 
training  system  could  be  through  simple 
identifiers  without  requiring  the  training 
system  to  store  the  speech  material. 

The  speech  response  requirements  are 
summarized  in  Table  I. 

Speech _ Recognition.  We  have 

discussed  the  requirements  for  the 

voice-response  capability  of  the  CAI 
system.  The  requirements  on  the 

speech-recoqnition  side  are  less 

substantial.  Speech-recognition 

capability  in  combination  with  voice 

response  would  be  very  powerful  if  the 
speech-recognition  capability  could 

distinguish  the  following  commandsi 


"Repeat"  — 

The  student  could  use  this  to  request 
that  course  material  or  a  question  be 
repeated. 


"Hint"  or  "Help"  — 

The  student  could  use  this  to  request 
further  information  or  a  hint  to  the 
correct  response. 


"Yes"  and  "No"  — 

To  allow  the  student  to  respond  to 
questi ons. 


"Stop"  and  "Ready"  — 

To  allow  the  student  to  stop  the  CAI 
process  or  to  inform  that  he  or  she 
is  ready  to  proceed  again  after  an 
interruption. 


In  addition,  in  specific  applications,  it 
might  be  useful  for  the  student  to  respond 
by  a  series  of  digits  or  by  other  specific 
responses  to  inquiries  by  the 
voice-response  unit. 

If  these  requirements  could  be  met  at  a 
price  commensurate  with  the  cost  of  the 
trainer  and  justified  by  the  service 
provided,  it  is  quite  likely  that  the 
device  would  provide  an  effective  aid  in 
assisting  the  instructor  in  training  for 
certain  types  of  skills. 

TRAINING  DEVICES 

Another  type  of  training  which  is 
receiving  growing  interest  in  the  military 
and  elsewhere  is  the  use  of  dedicated 
gener al -purpose  training  devices.  Such 
devices  include  the  following!  (1)  general 
CAI  systems  based  upon  minicomputers  or 
mi crocomputers,  and  (2)  interactive  video 
systems,  both  videotape  and  videodisc.  In 
p  ar t i cul ar ,  mi crocomputer — control  1 ed 
videodisc  systems  hold  great  promise  for 
hiqhly  powerful  interactive  training 
systems  at  a  moderate  cost.  A  major 
disadvantage,  however,  of  CAI  systems  in 
general,  and  of  videodisc  systems  in 
particular,  is  the  difficulty  of  changing 
such  systems  once  the  program  is  designed. 
To  change  a  videodisc,  for  example,  may 
require  creating  a  new  master. 

This  problem  could  be  minimized  through 
the  use  of  an  adjunct  voice-response 
system  controlled  by  the  same 
microcomputer  that  controls  the  CAI  or 
videodisc  system.  Thus,  while  a  given 
frame  is  on  the  screen.  the 
speech-response  system  could  be  instructed 
by  the  microcomputer  to  speak  a  given 
section  of  material.  The  speech  material 
could  differ  depending  on  the  trainee’s 
response. 

The  same  considerations,  with  respect  to 
being  able  to  change  the  material  easily, 
apply  to  this  type  of  system  as  they  did 
to  the  simulators  and  part-task  trainers 
in  the  previous  section.  It  is  almost  an 
axiom  that  there  will  be  errors  in  a  given 
set  of  courseware  no  matter  how  often  it 
is  checked  before  being  committed  to 
software  or  to  videodisc;  a  related 
postulate  is  that  the  course  material  will 
become  outdated  in  part  as  soon  as  it  is 
finalized.  Thus,  the  voice-response  system 
requirements  of  Table  I  are  relevant  to 
the  present  section. 

In  the  area  of  the  supportive  speech 
recognition  devices,  however,  there  are 
different  considerations.  We  do  not  have 
the  requirement  in  most  cases  of  physical 
fidelity.  Therefore,  there  is  no 
difficulty  in  allowing  the  trainee  to 
communicate  with  the  CAI  system  through 
the  use  of  a  keyboard  or  other  means  such 
as  a  touch-sensitive  screen,  minimizing 
the  need  for  speech  recognition. 
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Table  It 

Requirement*  -for 
Computer-to-Tr ai  nee 
Voice  Response 

A.  Easy-to-under  ,.,tand  speech 

B.  Easily  chancjud  speech  material 

C.  Unconstr ai nwd  vocabulary  and  syntax 

D.  Different  spoken  material  depending 

on  student  action,  with  no 

significant  '  time  delay  before 
response  starts 

E.  At  least  fifteen  minutes  of  speech, 
one  hour  desirables  course  material 
chanqed  by  reloadinq  from  off-line 
storage  (e.g.,  tape) 

F.  Sound  effects  and  different  voices 
possible  (useful,  but  not  important) 

G.  Minimal  impact  on  traininq  system 
hardware  and  software 


There  would  be  a  significant  motivation 
for  usinq  a  speech-recognition  system  as  a 
response  mechanism  if  the  response  of  the 
user  required  a  minimum  of  familiarization 
by  the  trainee  with  the  speech  system; 
that  is,  if  the  speech  system  took 
advantage  of  the  naturalness  of  speech  to 
the  user  as  a  means  of  communication,  it 
miqht  make  the  interaction  with  the  CAI 
system  less  i nt i mi dati nq .  It  would  be 
nice,  for  example,  if  the  videodisc  or  CAI 
system  could  display  a  multiple-choice 
question  on  the?  screen  and  ask  t.he  user  to 
repeat  the  correct  response.  The  response 
choices  would  ideally  be  lenqthy  phrases 
with  no  vocabulary  constraints.  If  the 
system  could  distinguish  the  correct  from 
the  incorrect  responses,  this  would  be  a 
very  natural  means  of  interacting  with  the 
students.  It  would  have  the  added  value 
of  havinq  the  student  repeat  the  correct 
answer  orally,  rather  than  simply  press  a 
button  marked  A,B.C,  or  D.  It  is 
reasonable  to  assume  that  a  student  would 
better  retain  an  answer  which  required 
oral  repetition.  This  would  seem  on  the 
face  of  it  to  be  an  easy  task  for  a 
speech-recognition  system,  since  the 
phrasing  of  the  multiple-choice  answers, 
particularly  the  wrong  answers,  is  very 
much  under  the  control  of  the  desiqner. 
It  would  theref ore  seem  to  be  easy  to 
select.  choices  which  are  distinctly 
di f f erent . 

In  the  next  section,  we  discuss  the 
implications  for  speech-recognition 
technology  of  the  requirements  of  this 
section  and  the  previous  section. 

IMPLICATIONS  FOR  SPEECH-RESPONSE 

AND  SPEECH-RECOGNITION  TECHNOLOGY 

There  are  two  distinct  technologies 
involved  in  this  discussion;  1)  voice 
response,  and  2)speech  recognition. 


Voice  Response 

There  are  four  major  approaches  to  speech 
synthesis;  (1)  analog  recording,  (2) 
off-line  encoding,  (3)  phoneme  synthesis, 
and  (4)  waveform  coding.  In  this  section, 
we  will  discuss  these  tr rhnologies  and  the 
degree  to  which  they  can  meet  the 
requirements  of  Table  I. 

Analog  Recording.  Because  they  are 
not  well-suited  for  allowing  interactive 
responses,  analog  (tape-  or  drum-based 
approaches)  are  not  attractive  for 
voice-interactive  systems.  However,  if 
one  wishes  to  sacrifice  voice  interaction, 
one  can  use  a  standard  cassette  system 
with  tones  indicating  the  next  CAI  or 
videodisc  segment.  For  some  applications, 
this  may  be  cost-effective;  as  a 
general -purpose  interactive 
speech-response  system,  it  is  not. 

Off-Line _ Parameter _ Encoding . 

Techniques  such  as  linear  predictive 
coding  (l.F'C)  are  used  to  analyze  a  spoken 
word  or  phrase  and  reduce  the  storaqe 
requirements  for  storinq  that  word 
diqj tally.  The  encodinq  is  done  off-line, 
one  word  or  phrase  at  a  time,  on  a  system 
different  from  the  system  which 
synthesizes  the  speech.  The  synthesis 
system  can  be  inexpensive.  The  resulting 
speech  quality  is  related  to  the  amount  of 
data  reduction  produced  by  the  coding, 
but,  in  qeneral ,  the  result  is  easy  to 
understand.  This  technology  is  suitable 
for  short  responses  and  material  which 
does  not  require  changes;  but  the 
difficulty  of  changing  the  speech  and  the 
large  amount  of  speech  required  eliminate 
this  approach  from  practical  consideration 
for  the  applications  of  this  paper. 


Phoneme  synthesis.  One  can  describe 
a  phoneme  synthesis  system  roughly  as  a 
device  which  pronounces  each  letter  of  a 
word  which  is  spelled  out.  The  speech  can 
be  entered  as  a  string  of  "phonemes" 
(analoqous  to  letters)  and  will  be 
pronounced  as  entered.  A  great  deal  of 
effort  is  required  to  enter  phonemes, 
pauses,  and  stresses  so  that  the  resulting 
speech  sou, ids  reasonably  natural;  even 
with  effort,  the  speech  is 

"robot-sounding,"  and  the  listener  must 
expend  some  effort  to  understand  it.  It 
is  feasible  to  use  phoneme  synthesis  for 
the  traininq  applications  of  this  paper; 
but  this  technology  places  a  burden  on  the 
trainee  in  understanding  the  speech,  and 
changing  the  material  can  oe  difficult  and 
time-consuming.  Text-to~speech  systems 

under  development  may  ease  this  latter 
difficulty  (2).  This  approach  conserves 
computer  memory  more 
approach,  so  a  great 
could  be  stored.  Rapid 
part  of  the  material 
This  could 
approach  to 


di sadvant ages  noted. 


than  any  other 
amount  of  material 
retrieval  of  any 
is  easily  possible, 
be  p  relatively  low-cost 
voice-aided  traininq  with  the 
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Waveform  coding.  This  technology 
basically  stores  a  replica  of  the  speech 
waveform  in  real  time  as  the  speech  is 
spoken.  Codinq  techniques  used  in 

communications,  such  as  Pulse  Code 
Modulation  (PCM)  codinq,  can  be  used  to 
reduce  memory  storaqe  requirement*.  It  is 
theoretical ly  possible  to  use  techniques 
such  as  L.PC  computed  in  real  time  to 
reduce  storaqe  even  further.  With  PCM 
codinq,  speech  data  can  be  stored  at  about 
4,000  bytes  per  second  to  create  hiqhly 
intelliqible  telephone-quality  speech. 
The  speech  can  be  qenerated  and  chanqed  by 
simply  speaking  into  a  microphone. 

Waveform  codinq  satisfies  all  the 

requirements  we  outlined  for  voice-aided 
traininq  in  Table  I.  It  has  the  further 
advantaqe  that  it  is  a  wel 1 -devel oped 
technology.  Unfortunately,  the  larqe 
memory  requirements  make  this  one  of  the 
more  costly  solutions!  one  half-hour  of 
speech  requires  over  seven  meqabytes  of 
storaqe.  On  the  other  hand,  Winchester 
disc  drives  of  ten  to  twenty  megabyte 


capacity  are  readily  available  and 
becoming  increasingly  inexpensive. 
Waveform  codinq  is  a  full  solution  to  the 
voice-aided  training  requirements  for 
voice  response  in  Table  I  —  althouqh  not 
an  inexpensive  solution. 


Speech  Recognition 

The  simple  control  words  required  by  the 
simulator  application  can  be  implemented 
with  any  commercial  isolated  word 
recognition  device.  Distinguishing  words 
such  as  "help,"  "repeat,"  etc.,  is  not 
difficult. 

An  isolated  word  recoqnizer  is  not  so 
suitable  for  distinguishing  multiple- 
choice  questions  of  essentially  unlimited 
vocabulary.  The  isolated  word  recognizers 
require  each  word  or  phrase  to  be  less 
than  two  seconds  or  so|  each  such 
word /phrase  to  be  used  must  be 
individually  spoken  several  times  by  the 
trainee  to  "train"  the  recognizer?  and  a 
long  multiple-choice  reponse  which  is  a 
series  of  shorter  word/phrases  requires  a 
distinct  pause  between  word/phrases. 
Isolated  word  recognition  is  a  complicated 
way  to  solve  what,  in  this  case,  is  a 
simple  problem. 

Because  of  the  flexibility  we  have  in 
choosing  the  multiple-choice  responses,  we 
can  choose  them  to  be  different  in  the 
number  of  "syllables"  (more  accurately,  in 
the  number  of  energy  pulses).  By  simply 
monitoring  the  enerqy  envelope  of  the 
trainee's  response,  the  speech  recognition 
system  can  distinguish  trainee  responses 
which  differ  in  energy  pulses.  Since  no 
spectral  information  is  required,  the 
system  is  speaker-independent  and  requires 
no  traininq.  Since  this  approach  cannot 
distinguish  "yes"  from  "no,"  it  does  not 
replace  isolated  word  recognizers  where 
distinctions  between  words  of  equal 
syllable  count  are  required. 


The  problem  of  countinq  energy  pulses 
consistently  over  many  users  is  more 
difficult  than  implied  here,  but  has  been 
demonstrated.  In  particular,  adding  this 
type  of  recognition  capability  to  a 
microcomputer-based  speech  response  system 
requires  a  minimum  of  additional  hardware. 

SUMMARY  AND  CONCLUSIONS 

If  one  wishes  to  use  CAI  with  training 
systems  which  replicate  part  of  a  weapons 
system,,  voice  response  and  speech 
recognition  allow  the  trainee  to  interact 
with  the  CAI  system  without  interfering 
with  the  fidelity  of  the  trainer. 

For  traininq  devices,  voice  response  can 
allow  chanqes  to  the  traininq  material  to 
be  made  easily  by  dictating  into  a 
microphone.  Speech  recoqni-ion  allows 
trainees  to  respond  to  the  system  in  a 
manner  more  comfortable  to  them  than  a 
keyboard  and  in  a  way  which  may  improve 
retention  of  course  material. 

It  is  feasible  to  have  all  the  key 
characteristics  implied  by  these 
applications  in  a  voice-response  system 
with  current  wavef orm-codi nq  technology. 
Because  of  the  memory  storage  required  by 
this  approach,  a  system  with  a  hard  disk 
drive  is  required?  although  such  devices 
are  becoming  smaller  and  declininq  in 
cost,  this  approach  is  relatively  costly. 
The  cost,  however,  is  low  compared  to  the 
cost  of  simulators,  part-task  trainers, 
and  their  instructors’  salaries.  The  cost 
is  not  so  low  compared  with  the  cost  of  a 
single  low-cost  traininq  device,  but 
perhaps  acceptable  if  a  sinqle  speech 
response  system  serves  some  ten  to  twenty 
training  devices. 

A  possible  lower-cost  alternative  is  a 
text-to-speech  system  using  phoneme 
synthesis.  Currently  the  quality  of  the 
resulting  speech  is  questionable?  but 
research  continues. 

Speech  recognition  has  been  discussed  in 
this  paper  as  a  supplement  to  voice 
response  rather  than  as  an  end  in  itself. 
This  approach  is  motivated  by  the 
realization  that  it  is  fairly  easy  and 
inexpensive  to  add  both  isolated  word 
recognition  (using  board-level  systems) 
and  speaker-independent,  phrase  recognition 
(syllable-counting)  to  a  microcomputer- 
based,  voi ce-response  system.  Except  for 
multi-user  environments,  the  speech 
response/recognition  system  should  never 
be  talking  while  it  is  listeninq,  or  vice 
versa?  thus,  a  sinqle  microcomputer  can 
control  both  voice  response  and 
recognition.  The  microcomputer  can  also 
handle  communications  with  the  training 
system  throuqh  a  standard  serial  interface 
to  minimize  any  impact  on  the  trainer 
hardware  and  software. 

In  implementing  a  total  system  (voice 
response,  two  types  of  speech  recognition, 
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and  communications  with  ths  trainer 
computer),  one  accomplishes  more  than  a 
cost  reduction.  The  speech  recognition 
and  response  capabilities  complement  one 
another  and  yield  a  highly  versatile 
system.  For  example,  the  system  may 
interact  with  the  trainee  by  giving  vocal 
instruction  and  requesting  a 

multiple-choice  response  which  is 

interpreted  by  the  syllable-counter.  The 
voice-response  system  can  then  ask  a 
question  which  requires  a  numerical 
response;  that  response  can  be  interpreted 
with  the  isolated  word  recognizer.  The 
isolated  word  recognizer  can  accept 
control  words  such  as  “help,"  “repeat,"  or 
"wai t . " 

The  technology  -for  effective  voice-aided 
training  is  available;  it  has  the 
potential  to  make  computer-aided 

instruction  more  practical  in  many 
appl i cations. 
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ABSTRACT 

Computer  speech  recognition  provides  the  enabling  technology  for  the  use 
of  automated  performance  measurement  and  instructor  support  features  to  allow 
" in struc torle ss"  training  for  those  tasks  which  are  primarily  speech  in 
nature.  The  design  guides  take  the  approach  that  automated,  "instructorless" 
training  is  achievable  through  the  use  of  computer  software  models  of  the 
instructor  and  the  task.  Human  factors  design  guidelines  are  provided  for  the 
integration  of  speech  technology  with  the  software  models.  The  Navy  has  built 
two  prototype  training  systems  using  voice  technology  to  capture  student 
behavior,  evaluated  one  of  them,  and  is  in  the  process  of  evaluating  another. 
Further,  the  training  implications  for  airborne  applications  of  voice 
technology  have  been  developed.  The  emergence  of  voice  technology  as  one 
solution  to  the  manpower  shortage  has  provided  justification  for  these 
efforts.  This  paper  describes  how  voice  technology  can  make  the  transition 
from  RAD  to  application. 


INTRODUCTION 

Background 

For  several  years,  the  Human  Factors 
Laboratory  at  the  Naval  Training 
Equipment  Center  has  been  developing 
techniques  for  the  application  of  voice 
technology  to  training  systems.  There 
has  emerged  from  that  work  an 
identification  of  three  areas  of 
expertise  which  will  be  required  in 
future  efforts  to  apply  voice  technology 
to  Navy  training  systems. 

The  first  area  is  Systems 
Engineering.  Obviously,  knowledge  is 
required  in  hardware  capability  and  in 
programming  requirements  of  voice 
technology  subsystems.  The  second  is 
Human  Factors.  Obviously,  too,  knowledge 
is  required  in  man-machine  interface 
design  requirements.  The  third  area  is 
relatively  new,  the  Voice  Technology 
Specialist.  Knowledge  is  required  in  the 
integration  of  Systems  concepts  with 
Human  Factors  design  concepts.  What 
makes  the  Voice  Technology  Specialist 
separate  is  the  requirement  to  recognize 
the  unique  characteristics  of  Voice 
Technology.  Talking  to  a  machine  with 
today's  technology  is  not  like  talking  to 
another  person.  Special  skills  are 
required  to  properly  Integrate  a  voice 
subsystem,  and  this  paper  will  address 
what  those  skills  are. 

Each  of  the  three  areas  of  expertise 
has  a  critical  role  to  play  in  the  design 
and  implementation  of  the  training  system 
using  voice  technology.  This  paper  seeks 
to  define  those  roles  in  terms  of  the 


tasks  which  must  be  performed.  It  is 
proposed  that  the  first  two  roles  will 
merge  to  form  the  third  as  more  and  more 
voice  subsystems  are  applied  to  training 
systems.  The  method  of  this  merger  is 
described  in  the  conclusion  of  this 
paper . 

Growing  Popularity 

Science  fiction  has  identified 
numerous  applications  for  voice 
technology,  but  the  technology  will  have 
to  be  able  to  converse  as  smoothly  as  the 
conversation  between  two  people  for  those 
applications  to  be  feasible.  Industry 
has  shown  cost  payoffs  with  today's 
technology  with  applications  in  the  data 
entry  and  the  secure  access  areas.  So, 
we  can  expect  to  see  a  slow  but  growing 
development  from  current  technology 
capabilities  to  "natural"  communication 
with  machines.  The  length  of  time  that 
it  takes  for  the  growth  will  depend  upon 
the  speed  at  which  voice  technology  i3 
popularized  by  reduced  cost  and  upon  the 
speed  at  which  need  is  created  for  more 
sophisticated  applications  of  man-machine 
dialogue.  The  Navy  is  shaping  this 
development  by  its  applications  of  voice 
technology. 

This  paper  will  briefly  describe  two 
Navy  training  prototypes.  They  have 
taught  us  much  about  the  unique  charac¬ 
teristics  of  voice  subsystems  and  have 
allowed  development  of  the  voice  technol¬ 
ogy  design  principles  discussed  here. 

The  two  prototypes  are  for 
controller  training.  The  Navy  has 
built  (1)  and  evaluated  (2)  a  system  for 
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training  a  portion  of  the  task  of  the  Air 
Traffic  Controller,  and  has  built  (report 
in  preparation)  and  is  evaluating  (work 
in  progress)  a  second  system  for  a 
portion  of  the  task  of  the  tactical 
controller,  the  Air  Intercept  Controller. 
In  each  case,  voice  technology  is  used 
within  a  package  of  instructional 
features.  Voice  technology  serves  to 
capture  the  vocal  behavior  of  the  trainee 
so  that  other  automated  subsystems  can 
evaluate  the  performance  of  the  trainee. 
Other  subsystems  then  select  the  next 
task  so  as  to  teach  the  trainee  the 
required  next  logical  skill.  This 
package  of  instructional  features  is  both 
the  reason  why  voice  technology  can  be 
made  to  work,  as  well  as  why  the 
application  principles  are  so 
complicated.  That  is,  a  non-trlvial 
dialogue  between  a  person  and  a  machine 
requires  an  intelligent  person  and  an 
intelligent  machine. 

Future  Payoffs 

The  Navy's  cost  payoff  from  voice 
technology  is  currently  in  the  area  of 
Manpower,  Personnel  and  Training.  Voice 
technology  as  part  of  an  instructional 
features  package  can  serve  to  substitute 
for  personnel  actions  and  in  some  cases 
for  personnel  themselves.  Thus,  fewer 
people  can  perform  more  tasks  in  less 
time  because  technology  is  performing  the 
functions  once  done  by  people.  In  the 
1990's,  voice  technology  combined  with 
artificial  intelligence  (AI)  is  expected 
to  pay  off  additionally  in  terms  of 
reducing  the  complexity  of  operating 
sophisticated  training  devices  30  that 
training  tasks  such  as  tactics  or  gaming 
are  i n s t r uc t i o n a  1 1 y  manageable  and 
performance  is  measureable.  This  will  be 
important  in  the  1990's,  even  if  the 
manpower  shortage  is  reduced. 

Organization  of  Remainder  of  Paper 

In  the  three  sections  which  follow, 
the  discussion  will  be  concerned  with 
tasks  required  of  the  Systems  Engineer, 
the  Human  Factors  Engineer  and  the  Voice 
Technology  Specialist.  The  Systems 
Engineer  will  be  required  to  design 
systems  in  terms  of  two  dynamic, 
interactive  models.  The  task  model 
generates  events  to  be  presented  to  the 
trainee.  The  instructor  model  delivers 
instruction.  The  voice  technology 
subsystem  must  then  be  designed  to  pass 
sufficient  data  to  the  models  to  allow 
the  student  to  Interact  with  them.  Of 
course,  a  model  can  be  composed  of 
submodels,  depending  upon  requirements. 

The  Human  Factors  Engineer  will  be 
required  to  design  the  feedback 
subsystem.  For  example,  people  can  use  a 
simple  frown  during  conversation  to 
indicate  a  word  wasn't  understood, 
However,  a  training  system  using  voice 
recognition  must  provide  a  functionally 
equivalent  source  of  feedback  that 


doesn't  distract  the  trainee  from  the 
task. 

The  Voice  Technology  Specialist  must 
integrate  these  two  systems.  Voice 
technology  data  capabilities  such  as 
indications  of  potentially  confusing 
phrases  or  hesitant  speech  must  be 
matched  with  an  appropriate 
instructionally  relevant  encouragement 
from  the  system  so  that  the  trainee 
receives  functionally  the  same  type 
instruction  a  teacher  would  deliver  in  a 
simlliar  situation. 


THE  SYSTEMS  ENGINEER 
The  Role  of  Models 

Supporting  software  models  will  be 
required  when  a  speech  recognition  device 
is  included  in  a  complex  Navy  training 
system.  Models  are  not  needed  when  voice 
is  merely  a  data  entry  device:  a 

keyboard  replacement.  The  real  power  in 
a  training  system  with  voice,  however  is 
achieved  through  the  relationship  between 
voice  recognition  and  two  major  models: 
the  model  of  the  task  and  the  model  of 
the  instructor.  As  an  example,  consider 
the  Ground  Controlled  Approach  Controller 
Training  System  (GCA-CTS),  designed  to 
provide  automated  instruction  of 
Precision  Approach  Radar  (1,  2).  The 
task  model  is  the  pilot/airoraft  model 
which  simulates  the  responses  to  the 
student  controller's  verbal  trans¬ 
missions,  such  as  "Turn  right  heading 
160."  A  closed-loop  system  is  achieved 
by  the  student  observing  the  simulated 
radar  scope  and  making  appropriate 
transmissions  which  are  "understood"  by 
the  voice  recognition  system,  resulting 
in  the  appropriate  changes  in  the  radar 
display.  In  this  case,  the  integration 
of  voice  recognition  with  the 
p i 1 o t / a i r c r a f t  model  has  enabled  a 
real-time  interactive  simulation.  An 
added  benefit  is  that  no  human  "pseudo 
pilot"  is  required  to  manually  simulate 
the  pilot  responses.  Therefore,  the 
combination  of  voice  recognition  and 
pilot/aircraft  modeling  ha3  achieved  a 
more  controlled  response  of  the  simu¬ 
lated  pilot  while  eliminating  the  need 
for  one  support  person  for  each  trainee. 
In  a  similar  development,  a  pilot/air¬ 
craft  model  has  been  designed  by 
Hooks  (3)  in  support  of  automated  train¬ 
ing  for  the  Landing  Signal  Officer  (LSO). 

Another  type  of  model  which  is 
closely  related  to  the  use  of  voice 
technology  for  training  is  the  instructor 
model.  Several  studies  have  dealt  with 
the  issue  of  how  to  move  toward 
"instructorless"  training  through  the  use 
of  software  models  of  the  instructor  and 
the  task  ( 4 ,  5).  Instructor  model 
functions  for  an  LSO  training  system  will 
soon  be  completed  (6).  Advances  in  the 
field  of  artificial  intelligence  (AI) 
promise  to  enhance  the  development  of 
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"intelligent"  instructor  models  (7),  and 
the  NAVTRAEQUIPCEN  currently  is 
initiating  work  on  this  issue. 

Instructor  models  will  never  totally 
replace  a  human  instructor  for  long 
periods  of  training,  but  instructorless 
(or  nearly  instructorless)  training  is 
feasible  now  and  can  provide  the 
capability  both  to  relieve  instructor 
manpower  shortages  and  to  promote 
effective,  objective,  consistent 
training.  Instructor  models  in 
voice-interactive  training  systems  must 
be  capable  of  several  functions, 
Including  the  following: 

•  Provide  Instruction 

•  Measure  and  Evaluate  Performance 

•  Provide  Performance  Feedback 
to  the  Student 

•  Decide  on  the  Appropriate 
Individualized  Instruction 
(remediation,  task  difficulty,  etc.) 

•  Keep  Records  of  Students'  Progress 

•  Communicate  Relevant  Information 
to  the  Human  Instructor. 

These  functions  are  not  limited  to 
training  systems  with  voice  technology, 
but  the  voice  capability  carries  with  it 
a  special  set  of  considerations  for  the 
design  of  the  instructor  model.  For 
example,  automated  instruction  can  take 
full  advantage  of  voice  interaction  by 
using  speech  generation  as  well  as  video 
display  to  demonstrate  proper  and 
improper  procedures.  Speech  recognition 
and  speech  generation  can  provide  the 
basis  for  a  natural  language  interface 
between  the  student  and  the  simulated 
instructor.  The  student  can  use  voice  to 
query  the  system,  and  ask  for  review  or 
additional  information.  Voice 
interaction  with  the  automated  instructor 
is  a  natural  communication  medium.  The 
focus  here  is  to  make  the  student  an 
active  part  of  the  instructional  process, 
rather  than  a  passive  recipient  of 
information. 

The  perfor.  nee  measurement  and 
evaluation  function  in  a  voice 
interactive  training  system  for  verbal 
tasks  will  be  critically  dependent  on  the 
accuracy  of  voice  recognition.  Accurate 
performance  measurement  for  air  traffic 
controller  training,  for  example,  can  be 
severely  degraded  by  speech  recognition 
errors.  Careful  design  of  the  supporting 
software  is  required  to  assist  in  the 
discrimination  between  a  student’s 
performance  error  and  a  speech 
recognition  error.  The  importance  of 
accurate  discrimi’'~tion  -?en  these  two 

types  of  e;  i  s  o'  is  when  'ne 

realizes  that  iiicorre*.  ,i,v  attributing  a 
speech  recognition  error  to  a  student 
error  can  be  carried  through  to  faulty 
performance  feedback,  syllabus  decisions, 
and  record  keeping.  Task  oriented 
software  based  on  AI  principles  appears 
to  te  a  promising  appr  i  to  this 
problem. 


The  instructor  model  also  may  be 
assigned  the  responsibility  of  managing 
the  support  requirements  of  the  voice 
recognition  subsystem,  such  as  collecting 
voice  reference  patterns,  providing 
instruction  on  "how  to  talk  to  the 
system,"  monitoring  confusion  matrices  to 
prompt  voice  retraining,  and  supporting 
voice  retraining  when  requested  by  the 
student.  These  functions  must  be 
accomplished  through  the  coordination  of 
the  instructor  model  and  the  voice 
subsystem.  The  design  goals  are  to: 
minimize  the  time  required  for  the 
student  to  learn  to  use  the  voice  system; 
avoid  long  and  tedious  voice  training 
(data  collection)  sessions;  and  assist 
the  student  in  maintaining  high 
recognition  accuracy  over  time.  This 
topic  is  discussed  later  under  Reference 
Pattern  Formation. 

A  student  model  is  another  candidate 
for  inclusion  in  an  automated  training 
system.  The  function  of  the  student 
model  is  to  generate  inferences  about  the 
changing  state  of  knowledge/skill  of  the 
student.  These  inferences  are  used  to 
select  appropriate  individualized 
instruction.  Adaptive  training  in  the 
form  of  either  multiple  syllabus  branches 
or  variable  task  difficulty  can  be 
supported  by  a  student  model,  as 
discussed  in  several  NAVTRAEQUIPCEN 
reports  (4,  5,  8).  The  design  of  the 
student  model  in  a  vo i ce- i nt e r ac t i v e 
system  is  not  particularly  unique,  other 
than  its  dependence  on  the  voice 
recognition  system  (and  the  performance 
measurement  system)  to  provide  the  input 
data  regarding  the  students’  current 
level  of  'verbal)  performance. 

Voice  Recognition  Issues  Affecting 
'Training  System  Design 

A  number  of  voice  technology  issues 
will  be  encountered  by  the  systems 
engineer  during  implementation  of  voice 
recognition  technology.  Although  there 
may  not  be  simple  answers  to  these 
issues,  Table  1  is  presented  in  the 
belief  that  identifying  some  of  the 
potential  problems  can  be  beneficial. 

Clearly,  the  list  of  issues  in 
Table  1  is  not  exhaustive.  It  is  merely 
a  sample.  In  general,  a  major  category 
of  issues  to  be  confronted  is  speaker 
variability  (fatigue,  sore  throat, 
lip-smacking,  non-meaningful  sounds  such 
as  "ah,"  amplitude  variation  with 
situational  stress,  etc.).  In  short,  all 
the  variabilities  that  make  speech  more 
interesting  (to  the  human  listener)  than 
a  predictable  monotone  are  what  make 
automated  speech  recognition  a  challenge. 
In  addition  to  speaker  variabilities, 
there  will  be  environmental  conditions  to 
be  confronted,  depending  on  the  training 
application.  Ambient  noise,  particularly 
of  the  impulsive  type,  may  be  a  factor. 
Consistent  microphone  placement  is 
important.  Motion  and  vibration  in  an 
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Table  1 


Issues  and  Suggestions  for  Implementing  Voice  Recognition 

ISSUES  POTENTIAL  SOLUTIONS/ SUGGESTIONS 

Vocabulary  Do  it  early,  ideally  during  a  full  ISD  process.  Consider 

Definition  stylization  requirements  and  natural  pauses.  Avoid 

confusible  items.  Consider  word/phrase  lengths  and  time 
required  for  speaking. 

Voice  System  Consider  objectives/requirements  of  training  system,  e.g., 
Selection  "real-time"  vocabulary  size,  isolated  vs.  connected  speech, 

and  sampling  requirements.  (See  9) 

Speech  Sampling  More  research  is  needed  on  opposing  viewpoints: 

(Voice  Data  (1)  Train  (sample)  in  random  order.  Seek  consistency 

Collection)  and  sampling  in  context  of  task. 

(2)  Sample  repetitively  on  the  same  word/phrase  seeking 
variability  in  reference  patterns.  Double-map  difficult  or 
alternative  words  (e.g.,  "nine"  and  "niner"). 

New  Users  A  "voice  recognition  test"  mode  should  be  available. 

Increasing  competence  (and  recognition  accuracy)  is  to  be 
expected  over  time  for  a  new  user. 

Recognition  Essential.  Alpha  numeric,  audio  or  situational  feedback 

Feedback  information  should  be  provided  immediately,  ideally  without 

interfering  with  the  speaker's  primary  task'. 

Maintenance  of  Automatic  or  manual  procedures  are  needed  to  facilitate 
Recognition  accuracy  maintenance.  Voice  test  function  is  needed.  Access 

Accuracy  to  "retraining"  is  needed.  A  confusion  index  is  recommended. 

"Understanding"  Supporting  software  (AI?)  is  recommended  to  enhance  speech 
Software  recognition  through  the  use  of  syntactic,  semantic,  and  task 

information . 

Training  System  All  software  "downstream"  of  speech  recognition,  such  as 
Integration  performance  measurement  and  adaptive  syllabus  control,  must 

be  designed  to  minimize  the  impact  of  speech  recognition 
errors . 

operational  training  environment  could  systems  o  r  i  e  n  t  e< d  t  o  c  o  m  p  i  ex  , 

introduce  additional  sources  of  variance.  technological  jobs  within  the  Navy.  The 

benefit  will  come  by  the  very 
"demystification"  of  automated  speech 
Navv  Training  Systems  recognition.  Ju3t  as  today's  recruits 

- — - - - -  find  hand  calculators  commonplace,  nearly 

The  two  prototype  Navy  training  every  Navy  recruit  in  1990  will  have 

systems  that  have  included  voice  operated  some  sort  of  voice  system.  This 

technology  have  been  relatively  will  tend  to  facilitate  user  acceptance 

sophisticated,  featuring  real-time  and  prevent  the  equivalent  of  "mike 

interactive  simulation,  automated  fright"  for  new  users.  The  challenge  for 

performance  measurement,  and  automated  the  training  system  designer  will  be  to 

instruction.  However,  simpler  voice  select  the  appropriate  voice  recognition 

applications  in  training  also  may  be  device  from  the  projected  large  number 

quite  worthwhile,  such  as  a  available,  and  adroitly  to  integrate  it 

voice-interactive  CAI  system.  One  can  into  the  training  system, 

envision  a  simple  CAI  system  with  a  very 

limited  vocabulary,  perhaps  consisting  of  Design  Guideline  Development 
eight  words  such  as  "ALPHA,  BRAVO, 

CHARLIE,  DELTA,  NEXT,  YES,  NO,  and  In  addition  to  the  implementation  of 

REVIEW."  Advances  in  the  technology  of  models,  the  systems  engineer  must 

integrated  circuits  reportedly  will  integrate  data  flow  between  the  models 

enable  a  single-chip  recognition  device  and  the  voice  technology  subsystem.  Some 

with  an  eight  word  vocabulary  to  be  of  the  issues  to  be  considered  in  that 

marketed  within  the  next  year  for  less  include  strengths  and  weaknesses  of 

than  $100.00  (10).  These  advances  in  various  features  of  speech  recognition 

hardware  technology  may  reduce  costs  to  systems,  such  as: 

the  point  of  popularizing  voice  •  Isolated  Word  Recognition  (IWR) 

technology  in  a  wide  variety  of  or  Connected  Speech  Recognition  (CSR) 

entertainment  and  consumer  products.  •  Speech  Stylization  Requirements 

This  presents  both  a  challenge  and  a  •  Speaker  Dependence/Independence 

benefit  to  the  designer  of  training  •  Vocabulary  Size 
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•  Voice  Data  Collection 

Number  of  Samples 
Procedures  for 

•  Voice  Test  and  Retraining 

•  Confusion  Matrices 

•  Recognition  Accuracy 

Misrecognition 

Non-recognition 

•  Recognition  Speed 

•  "Understanding"  Software. 

The  promised  advantages  to  be  gained 
from  voice  technology,  however,  can  only 
be  gained  by  careful  design  of  the  entire 
training  system  to  be  compatible  with  the 
capabilities  and  limitations  of  the 
trainee  ar.d  the  voice  subsystem.  The 
design  guidelines  currently  under 
development  (11)  are  intended  to 
consolidate  the  lessons  learned  from 
previous  prototype  system  developments, 
to  project  the  trends  of  voice 
technology,  and  serve  as  a  sourcebook  for 
specifying  design  options  and  choosing 
among  them,  based  on  the  objectives  of 
the  particular  training  system  being 
developed. 

There  are  numerous  physical, 
environmental,  and  human  factors  issues 
in  the  training  applications  of  voice 
technology.  An  "up  front"  Instructional 
Systems  Development  (ISD)  process  is 
necessary,  and,  at  the  other  end,  a 
generous  allocation  of  time  for  system 
test  and  "debugging"  is  required  to 
achieve  an  effective  voice-interactive 
training  system.  The  systems  engineer 
must  therefore  work  closely  with  ISD 
personnel  in  order  to  consider  all 
relevant  issues. 


HUMAN  FACTORS  CONSIDERATIONS 
The  Task  Analysis 

The  development  of  a  training 
device,  especially  a  major  device  for 
hands-on  training,  must  be  based  on  an 
analysis  of  the  tasks  or  skills  to  be 
trained.  This  training  task  analysis 
provides  the  basic  data  for  determination 
of  training  objectives  and  resulting 
training  system  performance  requirements. 
In  a  training  task  analysis,  the  emphasis 
should  be  on  relating  system  functions  to 
trainee  perceptions  and  responses, 
determining  special  skill  requirements, 
and  making  certain  that  each  system 
output  critical  to  task  performance  is 
linked  to  an  operator  response. 

One  of  the  recommendations  from  a 
recent  study  of  the  training  implications 
of  airborne  applications  of  voice 
technology  (12)  was  that  task  analyses 
and  other  front-end  analyses  for  training 
systems  employing  voice  technology  should 
be  performed  by  professional  personnel 
who  thoroughly  understand  the  human 
factors  of  voice-interactive  technology. 
Furthermore,  personal  hands-on  experience 
with  vo i c e- i n t e r ac t  i  v e  systems  was 
recommended  as  a  means  of  assuring 


familiarity  with  the  human  factors  of 
voice  technology.  Without  such  personal 
experience,  there  is  a  danger  that  some 
important  factors  in  learning  to  use 
automated  speech  recognition  and 
synthesis  will  not  be  reflected  in  the 
development  of  instructional  objectives. 

In  the  design  of  voice  technology 
for  systems  to  train  speech-based  tasks 
that  do  not  themselves  use  voice 
technology  --that  is,  in  using  voice 
technology  as  a  training  medium-- 
particular  care  must  be  taken  that  the 
constraints  imposed  by  voice  technology 
do  not  cause  the  training  tasks  to 
misrepresent  the  speech-based  tasks  being 
trained.  Personal  voice  technology 
experience  can  help  i n s t .  u c t i o n a  1 
designers  to  use  training  media  based  on 
voice  technology  in  ways  that  foster 
rather  than  hinder  accomplishment  of 
training  objectives. 

Reference  Pattern  Formation 

Most  of  the  presently  available 
automated  speech  recognition  systems  are 
"speaker-dependent",  that  is,  they 
require  that  each  operator  "train"  the 
system  by  providing  examples  of  that 
speaker's  pronunciation  of  the  words  to 
be  understood.  This  voice  reference 
pattern  formation  process  is  not  required 
for  speaker-independent  systems,  but  such 
systems  can  generally  recognize  only 
within  a  vocabulary  limited  to  a  very  few 
items . 

The  pattern  registration  process  in 
speaker-dependent  systems  requires  a 
speaker  to  pronounce  several  times  any 
word  or  phrase  to  be  recognized.  A 
composite  of  the  several  pronunciations 
is  stored  and  the  recognizer  system  can 
then  compare  future  utterances  with 
stored  composites  as  the  basis  of  its 
recognition  magic.  The  formation  of 
reference  patterns  is  extremely  important 
to  recognition  accuracy,  since  word  or 
phrase  recognition  occurs  when  an 
utterance  is  judged  by  the  computer  to 
match  one  reference  pattern  better  than 
any  other. 

It  is  to  be  expected,  then,  that  if 
a  word  or  phrase  is  spoken  in  a 
particular  way  during  reference  pattern 
formation,  and  then  spoken  differently 
later,  it  may  not  be  recognized 
correctly.  The  subtlety  of  differences 
which  can  interfere  with  recognition 
becomes  clear  only  after  one  has 
attempted  to  use  an  automated  speech 
recognition  device  using  today's 
technology.  Differences  which  are  not  at 
all  apparent  to  the  speaker  may  result  in 
non-recognition  or  misrecognition  by  the 
system,  leading  to  considerable 
frustration  on  the  part  of  the  user. 

Differences  between  the  context  in 
which  reference  pattern  formation  occurs 
and  in  which  recognition  is  attempted  can 
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result  from  changes  in  physical 
conditions,  such  as  noise,  vibration, 
G-forces,  and  other  factors.  The 
substantial  research  on  the  effects  of 
physical  context  has  been  summarized  by 
Coler  (13)  and  others.  Although  not  as 
well  studied,  differences  resulting  from 
changes  in  psychological  context  have 
often  been  noted  informally  by  many 
researchers  (14,  15). 

Among  the  most  widespread 
observations  are  1)  that  words  trained 
individually  may  not  be  recognized  when 
later  embedded  in  longer  utterances,  and 
2)  that  a  speaker  is  often  misrecognized 
when  speaking  in  a  stressful  situation  if 
the  voice  reference  patterns  have  been 
entered  in  a  non-stressful  setting.  Such 
misrecognition  may  be  self-perpetuating, 
since  it  induces  additional  stress,  w.iich 
leads  to  fu  ther  misrecognition.  If 
speech  recot  ition  is  to  work  well  in  a 
variety  of  psychological  contexts,  it  is 
probably  necessary  to  perform  voice 
reference  pattern  formation  under 
conditions  that  effectively  simulate  the 
range  of  operational  situations  to  be 
encounterec  .  It  may  be  possible  to 
obtain  spech  samples  which  are 
sufficiently  typical  of  the  trainee's 
normal  voicing  by  collecting  them  during 
the  practice  of  correct  terminology 
during  training,  as  was  done  for  parts  of 
the  GCA-CTS  vocabulary  (1). 

It  is  desirable  that  trainees  spend 
as  little  time  as  possible  in  speaking 
for  the  sole  purpose  of  registering  voice 
reference  patterns,  an  activity  with 
little  training  value  to  the  trainee. 
However,  if  voice  reference  pattern 
registration  is  well  integrated  into  the 
training  program,  it  can  occupy 
considerable  time,  and  that  time  will 
also  be  beneficial  to  the  trainee.  It  is 
essential  that  this  be  done  to  avoid 
wasteful  use  of  trainee  time,  and  also  to 
avoid  trainee  boredom  or  loss  of 
interest.  Fortunately,  such  integration 
of  reference  pattern  registration  into 
substantive  training  exercises  also 
serves  to  increase  the  likelihood  of 
proper  psychological  context  for 
reference  pattern  registration. 

Recognition  and  Re-training 

All  currently  available  Automated 
Speech  Recognition  systems  have 
performance  limitations  which  render  them 
less  efficient  and  less  adaptable  than  a 
human  listener.  Although  some 
limitations  are  not  easily  surmounted, 
others  stem  from  conditions  which  can  be 
controlled  to  minimize  their  detrimental 
effects  on  recognition. 

We  have  already  discussed  what  is 
perhaps  the  most  important  of  these 
controllable  factors,  which  is  the 
context  in  which  voice  reference  pattern 
formation  occurs.  Another  factor  known 
to  affect  recognition  accuracy  is 


variability  among  individual  users  in 
their  ability  to  "talk  to  a 
machine."  Some  users  are  consistently 
well  understood  by  Automated  Speech 
Recognition  devices,  while  others  have 
persistent  problems,  probably  in  part 
because  their  speech  is  more  variable 
(16). 

If  we  acknowledge  that  currently 
available  Automated  Speech  Recognition 
systems,  and  systems  likely  to  be  fielded 
in  the  near  future,  do  not  always 
recognize  speech  with  high  accuracy,  it 
becomes  necessary  to  assess  the  effects 
of  non-recognition  or  misrecognition  on 
the  performance  of  man-machine  systems. 
In  particular,  we  are  interested  in  the 
effects  on  the  user  and  on  interactions 
with  the  speech  recognition  system. 

In  the  design  of  training  systems 
using  voice  technology,  the  design  of 
system  feedback  on . recognition  accuracy 
is  critical,  requiring  exacting  human 
factors  analysis  for  each  specific 
application.  This  is  necessary  because 
recognition  failure  induces  frustration 
and  stress  in  the  user,  increasing  the 
probability  that  the  next  utterance  will 
be  misrecognized,  producing  further 
frustration  and  stress  in  a  vicious 
cycle.  To  break  this  cycle,  we  must 
provide  the  user  with  easily 
i n t e r p r e t a b 1 e  feedback  concerning 
recognition  accuracy  and  with  natural 
ways  of  responding  to  misrecognitions .  A 
well-designed  system  will  notify  the 
operator  that  it  has  not  understood,  or 
will  display  what  was  understood  to  have 
been  said,  thus  giving  the  speaker  a 
chance  to  detect  and  cope  with  a 
misrecognition  problem.  The  speaker  car, 
repeat  the  utterance,  or  retrain  the 
system  if  necessary.  A  system  which 
merely  fails  to  respond  appropriately  to 
an  utterance  will  leave  the  speaker  not 
knowing  what  has  gone  wrong,  nor  what  can 
be  done  to  set  it  right.  This  extremely 
frustrating  experience  can  have  a  strong 
negative  influence  on  the  user's 
acceptance  of  the  training  system  and 
attitude  toward  training.  Thus  the 
effectiveness  of  training  systems  using 
voice  technology  may  be  expected  to  vary 
directly  not  only  with  recognition 
accuracy  but  also  with  the  usefulness  of 
the  feedback  on  recognition  accuracy. 

The  voice  refe *ence  pattern  updating 
or  re-training  capability  of  training 
systems  using  voice  technology  must  be 
designed  with  reference  to  human  factors. 
Because  an  effective  training  program  for 
a  speech-based  task  will  lead  to  constant 
improvements  and  changes  in  a  trainee's 
speech  behaviors,  frequent  updating  of 
reference  patterns  may  be  expected.  This 
process  may  be  performed  openly,  in  such 
a  way  that  the  trainee  knows  that  it  is 
taking  place  (and  giving  the  trainee  a 
measure  of  control  over  the  process),  or 
it  may  be  integrated  into  the  training  30 
as  to  be  transparent  or  unnoticed  by  the 
trainee.  If  the  updating  is  transparent, 
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there  must  also  be  a  provision  whereby 
the  trainee  can  deliberately  test  and 
update  the  voice  reference  patterns  if 
poor  automated  speech  recognition 
performance  occurs  at  any  time  during 
training.  A  convenient  test  and  update 
capability  will  aid  in  preventing  the 
frustration  which  arises  from  incorrect 
recognition,  and  will  foster  trainee 
perceptions  of  control  over  the  training 
process.  It  will  help  prevent  an 
adversary  relationship  from  developing 
between  the  trainee  and  the  training 
system. 


Human-machine  Interface  for  V o  i  c  e 
Technology--Al ice ' s  Doorway 

In  the  report  on  training 
implications  of  airborne  applications  of 
voice  technology  (12),  an  analogy  was 
made  between  the  user  of  voice  technology 
and  Alice  in  Wonderland.  In  one 
adventure,  Alice  is  trapped  behind  a 
small  door  through  which  she  can  see  a 
marvelous  garden  with  bright  flowers'  and 
cool  fountains,  just  out  of  reach  beyond 
a  narrow  passageway.  Reaching  the  full 
benefits  of  voice  technology  may  also  be 
seen  as  requiring  a  user  to  traverse  a 
narrow  passageway,  a  channel  restricted 
by  the  human  factors  peculiar  to  current 
speaker-dependent  voice-interactive 
systems,  as  suggested  by  Figure  1.  Human 
engineering  of  the  man-machine  interface 
can  provide  the  "magic"  to  allow  the  user 
access  to  the  garden  of  benefits. 

As  Figure  1  shows,  the  human  factors 
that  must  be  reflected  in  the  design  of 
the  man-machine  interface  for  voice 
technology  are  not  limited  to  speech 
control  factors  resulting  from  the 
characteristics  of  present-day  automated 
speech  recognition  systems.  They  also 
include  general  factors  resulting  from 
use  of  auditory  and  voice  channels  for 
information  exchange  characteristic  of 
artificially  intelligent  systems. 


In  the  category  of  general  factors, 
one  of  the  more  difficult  problems  to 
deal  with  is  likely  to  be  resistance  by 
experienced  training  designers  and  users 
to  changes  induced  by  the  introduction  of 
voice  technology  in  training.  Heavy 
Involvement  in  the  design  of  the 
instructor  interface  by  personnel  who 
will  use  the  system  for  instruction, 
coupled  with  thorough  training  in  its 
use,  can  help  combat  this  resistance. 

Other  difficult  problems  stem  from 
teaching  users  the  reality  of  dealing 
with  limited  intelligence  machines,  and 
from  tendencies  to  view  the  machine  as  an 
adversary.  These  problems  can  be 
especially  troublesome  because  voice 
technology  systems  usually  do  not  provide 
feedback  and  verification  of  the  sorts 
people  expect  in  interaction  with  another 
intelligent  entity.  Simply  stated,  users 
may  attribute  too  much  intelligence-' •oo  a 
machine  just  because  it  talks  and 
listens,  and  they  can  become  angry  when 
it  fails  to  live  up  to  their 
expectations,  especially  if  the  basis  for 
its  misbehavior  is  not  readily  apparent. 
The  human-machine  interface  must  be 
designed  to  help  the  user  suppress 
inappropriate  responses  that  might  be 
directed  toward  another  human  and  to 
strengthen  appropriate  responses  that 
take  into  account  the  limitations  of  the 
machine . 

In  the  category  of  speech  control 
factors,  the  human-  machine  interface 
must  be  designed  to  aid  the  user  in 
dealing  with  any  constraints  on  user 
speech  patterns  peculiar  to  that 
automated  speech  recognition  system.  For 
example,  it  has  been  pointed  out  that 
recognition  accuracy  can  be  critically 
dependent  upon  characteristics  of  speech 
utterances  that  are  not  usually  attended 
to  by  a  speaker.  Feedback  on  user  inputs 
may  have  to  be  structured  to  help 
speakers  attend  to  such  subtle 
characteristics . 
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Figure  1 .  Human  factors  constraints  resulting  from  use  of  the  voice- 
interactive  channel  in  training  systems. 
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Voice  reference  pattern 
formation/updating  and  speech  recognition 
performance  limitations  also  have  an 
impact  on  the  design  of  the  human-machine 
interface.  Even  for  a  system  in  which 
voice  reference  pattern  formation  and 
updating  are  sequenced  within  training 
program  content  so  that  little  or  no 
trainee  time  is  spent  solely  for  those 
activities,  recognition  failures  may  be 
perceived  by  the  trainee  as  correctible 
by  better  reference  pattern  formation. 
To  maintain  user  acceptability,  the 
system  must  permit  a  user  the  option  at 
frequent  intervals  or  at  will  of  voice 
recognition  test  and  reference  pattern 
update. 

If  the  human-machine  interface  for 
voice  technology  in  training  systems  is 
engineered  with  the  user's  capabilities 
and  limitations  as  a  guide,  the 
restrictiveness  of  the  voice-interactive 
channel  can  be  minimized.  Human  factors 
engineering  and  a  well-designed  training 
program  can  provide  the  "magic"  to  allow 
the  user  access  to  the  garden  of  benefits 
made  possible  by  creative  application  of 
voice  technology. 

VOICE  TECHNOLOGY  SPECIALIST 

The  emergence  of  a  Voice  Technology 
Specialist  is  required  for  success  in 
complex  applications  and  high  payoff  in 
voice  technology.  To  encourage  that 
trend,  a  short  course  is  being  developed 
at  the  Naval  Training  Equipment  Center, 
Human  Factors  Laboratory,  to  provide 
hands-on  experience  with  voice 
technology.  It  is  expected  to  be  ready 
for  evaluation  early  in  1982  by  a  select 
group  of  Naval  Training  Equipment  Center 
instructional  systems  design  personnel, 
then  to  be  offered  Government-wide. 

Skills  of  the  Voice  Technology  Specialist 

Models .  Training  tasks  to  which 
voice  technology  can  be  applied  are 
characterized  by  requirements  for  the 
trainee  to  process  information,  manage 
data,  and  make  decisions.  The  model  of 
the  task  which  must  generate  the  data 
required  for  proper  training  is 
characterized  by  requirements  for 
generation  of  variability.  For  example, 
a  pilot/c.ircraft  model  for  a  controller 
trainee  must  be  capable  of  flying  various 
approach  profiles  which  exhibit  typical 
situations.  The  voice  technology 
specialist  must  ensure  that  the  model  is 
properly  integrated  with  the  voice 
subsystem  so  that  the  decisions  of  the 
trainee  as  spoken  to  the  system  are 
accurately  reflected  in  the  response  of 
the  model.  For  example,  voice 
recognition  systems  often  confuse  "port" 
with  "four"  (they  may  sound  different  to 
you  and  me,  but  electronic  ears  hear 
things  differently).  Thus,  the 
integration  task  of  the  voice  technology 
specialist  requires  identification  of 
potential  recognition  confusions  unique 


to  the  particular  hardware  and  vocabulary 
being  used.  Then,  logical  substitutions 
can  be  made  by  the  system  itself  when  a 
trainee's  decision  doesn't  make  sense. 

Similarly,  the  instructor  model  must 
evaluate  decisions  of  the  trainee  and 
provide  appropriate  feedback,  remedial 
training,  and  critique.  The  voice 
technology  specialist  must  ensure  that 
the  model  is  properly  integrated  with 
both  the  task  model  and  speech  subsystem 
so  that  the  trainee  doesn't  become 
confused  or  frustrated.  For  example, 
confusion  resulted  in  the  GCA-CTS 
prototype  system  during  a  particular 
two-part  maneuver.  The  first  part 
require^  a  number  of  decisions  in  a 
relatively  short  time.  If  any  were  not 
recognized  precisely,  the  trainee  was  not 
told  why  and  also  could  not  proceed. 
Since  the  task  was  timed,  there  occurred 
a  build-up  of  stress  which  reduced 
further  the  trainee's  chance  of  being 
understood.  Thus,  the  integration  task 
of  the  voice  technology  specialist 
requires  identification  of  potentially 
stressful  situations.  Then,  the  voice 
recognition  subsystem  can  be  designed  to 
relax  recognition  accuracy  requirements 
until  the  trainee  becomes  proficient  at 
the  task. 

Voice  Technology.  Certain  areas 
within  voice  technology  are  constantly 
improving  and  changing.  The  voice 
technology  specialist  must  keep  abreast 
of  developments  in  speaker 
d e p e n d e n t / i n d e p e n d e n t  systems, 
isolated/connected  word  recognition, 
vocabulary  size  vs,  system  cost 
trade-offs,  accuracy  of  recognition,  and 
software  access  to  voice  recognition 
parameters  such  as  a  confusion  matrix, 
threshold  values,  and  timing  of  incoming 
speech. 

Human  Factors .  One  area  of  human 
factors  has  been  concerned  with  training 
panel  layout  design.  In  the  voice 
technology  area,  however,  the  other 
design  criteria,  discussed  earlier  in 
this  paper,  become  important.  In 
particular,  there  is  a  learning-to- 
talk-to-the-machine  phenomenon  in  which 
most  people  can  achieve  99 . 9t  recognition 
accuracy  with  a  few  hours  practice.  What 
appears  to  happen  is  that  a  person 
develops  consistency  in  speaking  a 
particular  phrase  and  develops  uniqueness 
in  how  that  one  is  said  versus  how 
another  acoustically  similar  phrase  is 
said.  Exaggeration  of  consonants  is  the 
simplest  technique. 

The  voice  technology  specialist  must 
analyze  the  application  for  critical 
voice  technology  features,  then  optimize 
the  man-machine  interface  design  for  user 
friendliness.  Stress  is  particularly 
important  to  consider  in  training  system 
design.  The  higher  the  stress,  the  lower 
the  likelihood  of  recognition.  In 
training,  stress  often  comes  about  from 
poor,  erroneous,  or  no  feedback,  or  from 


too  fast-paced  responding.  The  voice 
technology  specialist  must  design  a 
system  that  combines  the  "intelligence" 
of  the  task  and  instructor  models  with 
the  recognition  parameters  of  the  voice 
subsystem  to  produce  a  dynamic, 
interactive,  voice-based  training  system. 

CONCLUSION 

This  paper  has  concerned  itself  with 
technology  transfer.  An  emerging 
technology,  computer  voice  technology, 
has  been  described  in  terms  of  the  skills 
required  to  apply  the  technology  to 
real-world  training  situations.  Transfer 
to  engineering  is  being  conducted  via 
development  of  design  guides  for  system 
models  and  subsystems.  Transfer  to 
instructional  systems  development  is 
being  conducted  by  development  of  Human 
Factors  principles  of  feedback  and 
exploitation  of  available  voice  quality 
data  from  the  system.  A  Voice  Technology 
Specialty  appears  to  be  emerging  from  a 
combination  of  the  System  Engineering 
skills  and  Human  Factors  skills.  This 
emergence  is  required  for  continued 
application  of  voice  technology. 
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ABSTRACT 

Increasing  systems  sophistication  and  limited  personnel  resources  are  dramatically 
demonstrating  the  need  for  cost  effective  training,  operations,  and  maintenance  assistance  at  the 
work  site.  The  Interactive  Multi-Media  System  (IMMS)  addresses  these  needs  by  providing 
training,  technical  documentation,  and  operations  Information  In  a  highly  Interactive  format 
using  advanced  microcomputer  and  optical  videodisc  design,  IMMS  uses  Spatial  Data  Management  as 
a  means  to  provide  users  with  rapid  and  natural  access  to  varied  types  of  Information  in  an  easy- 
to-use  and  unified  format.  There  are  no  languages  to  learn  or  typing  required  with  IMMS. 
Optical  videodiscs  are  used  In  IMMS  to  provide  users  with  Immediate  access  to  Interactive  video- 
based  Information.  Some  of  the  Interactive  videodisc  functions  supported  by  IMMS  Include 
diagnostic  procedures,  step-by-step  assembly,  maintenance,  and  operations  activities,  end 
technical  Information  support.  Other  multi-media  aspects  of  IMMS  Include  the  use  of  variable 
data  sources  such  as  microfiche,  videotape,  remote  data  systems,  and  teleconferencing  to  support 
changes  In  technical  Information  and  procedures  as  well  as  off-site  Instruction.  Finally, 
embedded  subsidiary  incentives  in  IMMS  stimulate  the  learning  and  usage  of  IMMS  by  users. 

INTRODUCTION 


The  productivity  limits  of  our  current 
training  technology  may  have  been  reached.  Massive 
efforts  to  Improve  "stand-up”  lectures,  printed 
training  materials,  and  "hands-on"  laboratory 
experience  have  begun  to  yield  too  few  significant 
returns.  We  need  revolutionary  new  techniques  that 
will  break  through  the  constraints  posed  by 
existing  training  technologies  and  allow  us  to  beat 
the  current  tradeoffs  that  must  be  made  among 
costs,  quantity  and  quality.  Videodisc  technology 
Is  one  of  the  most  promising  sources  of  these  new 
techniques. 

Videodisc  technology  is  creating  opportunities 
for  new  kinds  and  forms  of  training  devices  at 
surprisingly  low  cost.  At  the  heart  of  this 
technology  Is  the  capability  to  access  tens  of 
thousands  of  color  Images,  Including  stereo  sound. 
In  seconds  of  fractions  thereof.  When  coupled  with 
the  new  unlqultous  microprocessor,  videodisc 
technology  offers  new  opportunities  for  training 
applications  as  well  as  new  Issues  for  the  training 
special  1st, 

In  this  paper  we  share  some  of  our  ideas  and 
experiences  In  the  application  of  IMMS  to  training. 
Specifically,  we  will  examine  three  new  Ideas  for 
training  app  I  i  cat  I  oiis  which  use  videodisc 
technology.  We  then  consider  issues  relevant  to 
these  applications  and  their  Implications  for 
training.  We  begin  a  brief  review  of  the 
characteristics  and  capabilities  of  the  optical 
videodisc. 

OPTICAL  VIDEODISC  TECHNOLOGY 

A  videodisc  Is  similar  to  an  audio  record 
except  that  each  side  of  a  12- inch  disc  contains  30 
minutes  of  television.  Since  television  Is  the 


rapid  presentation  of  pictures  at  a  rate  of  30 
Images  per  second,  each  side  of  a  videodisc 
contains  54,000  still  pictures  In  color. 

The  player  for  a  videodisc  Is  much  like  a 
turntable  except  that  under  computer  control  the 
player  can  rapidly  find  any  particular  part  of  the 
videodisc.  Specifically,  any  one  of  the  54,000 
Images,  or  any  part  of  the  30  minutes  of  television 
on  a  videodisc,  can  be  located  typically  In  a 
fraction  of  a  second.  A  videodisc  provides  a 
combination  of  moving  and  still  Images,  any  part  of 
which  can  be  quickly  located. 

Videodiscs  are  made  like  audio  records  using 
original  materials  that  can  be  movies,  videotapes, 
pages  of  text,  tables  of  numbers,  graphs,  charts, 
maps,  drawings,  diagrams,  or  photographs.  From  the 
original  materials,  a  master  disc  Is  produced  that 
in  turn  Is  used  to  "press"  multiple  copies  speedily 
and  at  low  cost. 

Three  types  of  videodiscs  and  videodisc 
players  are  now  available.  The  players  are 
character  I  zed  by  the  market  they  are  targeted  for: 
general  consumer  usage  and  Industrial/educational 
applications.  Various  potential  manufacturers  such 
as  JVC,  IBM,  Xerox,  Zenith,  to  name  a  few,  are 
welting  In  the  wings,  but  only  six  are  now 
marketing  videodisc  players. 

-  Magnavox,  a  wholly  owned  subsidiary  of  Philips, 
began  selling  a  consumer  model  player  for  about 
$800  In  December  1978. 

-  Dlscovislon  (DVA),  under  license  by  MCA,  began 
general  sales  of  an  Industrial  player  for  about 
$3,000  In  June  1979. 
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-  Thompson-CSF  began  sales  of  an  Industrial  player  laser  focused  on  the  track  and  thereby  generate  a 

for  about  $ 3,500  early  In  1980.  signal  that  Is  processed  and  passed  to  a  standard 

video  monitor  (l.e.,  to  the  antenna  terminals  of  a 

-  Pioneer  began  sales  of  a  consumer  player  for  TV  set), 
about  $700  late  In  1980. 

One  video  frame  Is  stored  on  each  track  and 

-  RCA  began  sales  of  their  consumer  videodisc  there  are  54,000  tracks  per  disc.  Video,  audio, 

player  early  In  1981.  and  still  photographic  Information  can  all  be 

Intermingled  on  these  discs.  These  videodisc 

-  Sony  began  limited  sales  of  an  Industrial  systems  effectively  provide  rapid  access  to  30 

videodisc  player  In  mld-1981.  minutes  of  video  Information,  30  minutes  of 

analogue  audio  Information,  54,000  still 
Some  of  the  aforementioned  companies  have  announced  photographs,  wel  I  In  excess  of  400  hours  of 

marketing  plans  for  their  videodisc  systems.  digitized  audio  Information,  30  minutes  of  motion 

Matsushita  (which  markets  under  brand  names  such  as  picture  Information,  or  various  combinations  of  the 

JVC,  Panasonic,  and  National)  and  several  other  above.  The  point  to  be  made  Is  that  videodiscs 

Japanese  companies  have  announced  marketing  dates  provide  rapid  random  access  to  a  lot  of  Information 

of  first  quarter  1982  for  their  consumer  videodisc  which  can  be  Inexpensively  stored  and  replicated, 

systems.  Table  II  presents  a  summary  comparison  of  the 

videodiscs  discussed  In  this  section. 

The  differences  between  consumer  and 

industrial  systems  are  significant  as  Table  I  There  are  at  least  two  important  differences 

shows.  For  Instructional  settings,  the  Industrial  between  the  OVA/Sony/Phl I Ips  and  the  Thompson-CSF 

players  Incorporate  microprocessor  control  for  fast  videodisc  systems.  First,  the  Thompson  Disc  Is 

random  access,  pre-programmed  branching,  and  frame  "transmissive"  whl  le  the  others  are  "ref  lectlve". 

selection.  Microprocessors  have  also  been  Light  from  a  laser  passes  through  the  Thompson-CSF 

Installed  In  the  less  expensive  consumer  players  to  disc  and  Is  picked  up  on  the  other  side.  On  a  DVA, 

provide  random  access,  branching,  and  frame  Sony,  or  Philips  system,  the  disc  reflects  light 

selection.  However,  the  servo-mechanism  In  these  back  up  so  that  light  is  sent  from  and  received  on 

players  requires  from  15  to  20  seconds  to  locate  a  the  same  side  of  the  disc.  Three  I  mpl  I  cat  Ions  of 

frame  In  the  worst  case.  Maximum  access  times  for  this  difference  In  technical  approach  are:  (1)  the 

the  Industrial  players  are  under  four  seconds.  The  Thompson-CSF  disc  Is  "floppy".  It  can  be  rolled  up 

consumer  players  lend  themselves  to  Instructional  In  magazines  and  newspapers,  and  tha  reflective 

application,  but  the  functional  capabilities  of  the  disc  Is  rigid;  (2)  by  adjusting  the  focus  of  the 

Industrial  players  may  be  sufficiently  greater  than  laser,  either  side  of  the  Thompson-CSF  disc  can  be 

their  higher  cost  Is  justified.  With  experience,  read  without  physically  turning  the  disc  over.  If 

we  should  dl1-  over  what  various  videodisc  features  both  sides  of  the  reflective  disc  are  to  be  used, 

buy  In  terms  of  Instructional  achievement  and  will  It  must  be  physically  turned  ovei —  or  there  must 

thereby  be  able  to  recommend  commercial  or  be  a  light  source  for  each  side;  and  (3)  the 

Industrial  players  depending  on  Instructional  Thompson-CSF  disc  Is  more  sens  It Ive  to  dust  and 

settings.  dirt  than  the  reflective  disc. 


Features 

Consumer  Players 

Industrial /Educat I onal 
Players 

Cost 

$800 

$2300-53500 

Still  flame 

Manual  or  automa¬ 
tic 

Manual  or  programmed* 

Frame  random 

Manual 

Manual  or  programmed 

access 

Visual  idensiti- 

Visual , 

cation 

keyboard  selection,  or 
programmed 

Frame-by-frame 

"stepping" 

Manual 

Manual  or  programmed 

Variable  speed 
motion  (forward 
or  reverse) 

Manual 

Manual  or  programmed 

Two  discrete 
sound  channels 

Yes 

Yes 

Programmed  control 

may  be  through  an  on- 

-board  video  disc  player 

microprocessor  or 

an  external  computer. 

Table  1.  Comparison  of  Consumer  With  Industrial  Video  Discs 


With  the  exception  of  the  RCA  system,  the  The  second  I mportant  d I f ference  between  the 
videodisc  players  discussed  above  are  all  optical,  two  discs  Is  that  the  Thompson-CSF  disc  systems 
laser-based  systems.  They  use  a  12-Inch  disc  with  observe  European  PAL/SECAM  television  standards  as 
a  spiral  track.  The  track  Is  pitted  with  oblong  well  as  U.S.  standard  NTSC.  The  reflective  disc 
depressions  or  mlcroplts  about  1  micron  deep  that  observes  only  American  NTSC  televl slon  standards, 
vary  In  accordance  with  the  audio  or  video  when  used  In  PAL/SECAM  mode,  the  Thompson-CSF  disc 
Information  they  represent.  Dur  I  ng  p  I  ayback  l’ie  therefore  provides  more  lines  per  display.  The 
disc  spins  at  1,800  revolutions  per  minute  while  major  Instructional  Implication  of  this  difference 
these  mlcroplts  modulate  a  low  power  helium-neon  seems  to  be  that  because  of  the  better  resolution 
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.  . A- 


Magnavox/Phillips 

Dis covision  DVA 

Thompson-CSF 

Cose 

$800 

$2,fioo 

$3,300 

Market 

Consumer 

Indus  t  r  l  al  /I  du¬ 

Industrial /Edu¬ 

cat  ional 

cational 

Cost  of 

$  1 , 5003 

$1  ,300* 

$  1,300s 

Master 

Cost  of 

$5-$10C 

$5-$l0C 

$l8b 

Copies 

Technology 

Optical-reflective 

Optical-reflec¬ 

Opt  i  cal -t rans- 

(two-sided  alumin¬ 

tive  ( 

missive  (two- 

um  disc) 

sided  aluminum 

sided  plastic 

disc) 

disc) 

Standard 

NTSC 

NTSC 

NTSC,  PAL/SECAM 

a„  .  , 

Cost  per  side 

Cost  includes  $1.00  for  protective  plastic  cover 
c 

Depends  on  quantity. 


Table  II.  Summary  Comparison  of  Three  Video  Disc  Systems 


with  PAL/SECAM  standards,  programs  that  display 
large  amounts  of  text  may  be  better  suited  to  the 
Thompson  disc.  On  the  other  hand.  If  Instruction 
designers  want  to  take  advantage  of  the  millions  of 
already  purchased  television  sets  In  American 
homes,  schools,  and  Industries,  they  may  be  well 
advised  to  use  NTSC  encoded  videodiscs. 

Second  Generation  Disc  Technology 

Two  second  generation  developments  In 
videodisc  technology  appear  to  be  particularly 
notable  for  Instructional  applications. 

First  Is  the  anticipated  appearance  of  clrect 
read  after  write,  or  DRAW,  technology.  The 
Important  aspects  of  the  DRAW  disc  Is  that  It 
stores  10* 1 0 1 2  bits  of  digital  Information  on 
flach  side  of  the  d  I  sc.  This  development  Is  being 
undertaken  by  both  RCA  and  Phillips.  In  the  near 
term,  videodisc  systems  using  DRAW  technology  are 
likely  to  be  very  expensive.  However,  DRAW  disc 
systems  may  be  better  suited  for  the  small  volume 
experimental  development  that  is  characteristic  of 
many  Instructional  settings. 

Second  Is  the  development  of  still  frame 
sound.  All  currently  available  disc  systems  only 
provide  sound  when  the  disc  is  played  at  30  frames 
per  second.  Unfortunately,  this  detracts 
considerably  fror  the  disc's  Instructional  value 
during  slide-type  presentations.  DVA  and  Sony  have 
both  indicated  that  some  form  of  compressed  audio 
providing  3  to  30  seconds  of  sound  per  frame  Is 
under  development. 

jddeQdlSC  Versus  V I deotapn 

A  brief  comparison  of  videotape  capabilities 
with  those  of  videodiscs  may  be  In  order  at  this 
point.  After  al i,  videotapes  represent  a  rapidly 
maturing  technology,  and  they  can  offer  many  of  the 
features  of  videodiscs  such  as  random  access, 
variable  speed  play,  and  reverse  motion.  Three 
points  appear  to  be  salient: 

I.  Tapes  can  be  edited  and  current  videodiscs 
cannot.  This  difference  Is  likely  to  be  removed 
with  the  appearance  of  the  DRAW  technology 
discussed  above.  DRAW  technology  will,  of 
course,  provide  editable  videodiscs  quickly  and 


relatively  Inexpensively. 

2.  Random  access  to  arbitrary  points  on  a  videodisc 

Is  much  faster  than  similar  access  using 
videotape.  Random  access  on  industrial 

videodiscs  Is  less  than  4  seconds  compared  with 
20-100  seconds  on  videotape.  Moreover,  accuracy 
of  random  access  on  videotape  leaves  something 
to  be  desired. 

3.  Still  frames  are  possible  using  either  videodisc 

or  videotape,  but  It  Is  exceedingly  expensive 
and  cumbersome  on  videotape  and  single  stepping 
through  a  series  of  still  frames  Is  essentially 
Impossible.  A  far  more  comprehensive  and 

detailed  comparison  of  videotape  and  videodisc 
has  been  prepared  by  Houston  (2). 

APPLICATIONS  OF  IMMS  TO  TRAINING 

One  of  the  most  exciting  aspects  of  IMMS  Is 
that  It  makes  possible  an  entirely  new  set  of 
training  experiences.  This  section  describes  such 
new  applications  ranging  from  new  kinds  of  training 
movies  to  low-cost  simulators. 

I nteract 1  ye  Movies 

Interactive  Movies,  Illustrated  In  Figure  1, 
translate  movie  viewing  Into  an  active 
participatory  process.  In  effect,  the  viewer 
becomes  the  director  and  controls  many  features  of 
the  movie.  A  sampling  of  feature  controls 
available  to  the  viewer  Is  the  following: 

1.  Perspective.  The  movie  can  be  seen  from 
different  directions.  In  effect,  the  viewer  can 
"walk  around"  ongoing  action  in  the  movie  or 
view  It  from  above  or  below. 

2.  Detail.  The  viewer  can  "zoom  In"  to  see 
selected,  detailed  aspects  of  the  ongoing  action 
or  can  "back  off"  to  gain  more  perspective  on 
the  action  and  simultaneous  activity  elsewhere. 

3.  Level  of  Instruction.  In  some  cases,  the 
ongoing  action  may  be  too  rich  In  deta II  or  It 
may  Include  too  much  Irrelevant  detail.  The 
viewer  can  hear  more  or  less  about  the  ongoing 
process  by  so  Instructing  the  Interactive  Movie 
System. 


Fig.  I.  IKu&usiikiu  o!  an  Interact  i  vc 


4.  Level  of  abstraction.  In  some  Instances  the 
viewer  may  wish  to  see  the  process  being 
described  In  an  entirely  different  form.  For 
example,  the  viewer  might  choose  to  see  an 
animated  line  drawing  of  an  engine's  operation 
to  get  a  clearer  understanding  of  what  Is  going 
on.  In  some  cases,  elements  being  shown  in  the 
line  drawl ngs  may  be  invisible  In  the  ongoing 
action—  for  instance,  electrons  or  force  fields 
can  be  shown. 

5.  Speed.  Viewers  can,  of  course,  view  the  ongoing 
action  at  a  wide  continuous  range  of  speed- 
including  reverse  action  and  no  action  (st  1 1  I 
frame) . 

6.  Plot.  Viewers  can  change  the  "plot"  to  see  the 
results  of  different  decisions  made  at  selected 
times  during  the  movie. 

A  typical  application  for  Interactive  Movies 
would  be  In  training  (and  aiding)  equipment 
technicians.  The  technician  could  not  only  see  how 
a  particular  part  is  located  and  installed  from 
several  points  of  view  (e.g.  top  versus  bottom)  but 
could  interactively  control  how  detailed  a 
description  Is  either  seen  or  heard  regarding  that 
maintenance  activity. 

Several  Interactive  Movie  videodiscs  have  been 
completed  using  hand  to  hand  combat  (l.e.,  karate) 
as  the  subject  area.  These  discs  let  the  viewer 
not  only  control  playing  a  particular  karate  move 
backward  and  forward  at  any  rate,  but  also  include 
multiple  views  and  closeup  views  following  every 
move  from  four  different  positions.  In  progress 
are  several  Interactive  Movies  that  focus  on 
equipment  maintenance  tasks. 

Surrogate  Trayfll 

Surrogate  Travel,  Illustrated  In  Figure  2, 
forms  a  new  approach  to  local  familiarization  and 
low  cost  trainers.  The  basic  principle  Is  simple. 
On  videodiscs  are  stored  up  to  108,000  images 
showing  discontinuous  motion  along  a  large  number 
of  paths  In  an  area.  Under  microprocessor  control, 
the  user  accesses  different  sections  of  the  disc, 
simulating  movement  over  the  selected  path. 

The  user  sees  with  photographic  realism  the 
area  of  Interest.  Unlike  a  travel  movie,  the  user 
Is  able  to  both  choose  the  path  and  control  the 


Fi&.  2.  Illustration  of  Surrogatt  Iravel. 

speed  of  advance  through  the  area  using  simple 
controls.  The  videodisc  frames  the  viewer  sees 
originate  as  filmed  views  to  what  one  would 
actual  I y  see  In  the  area.  To  allow  coverage  of 
very  large  areas,  the  frames  are  taken  at  periodic 
I nterva Is  that  may  range  from  every  foot  Ins  I de  a 
building,  to  every  ten  feet  down  a  c I ty  street,  to 
hundreds  of  feet  in  a  large  open  area  (e.g.,  a 
harbor) . 

The  rate  of  frame  playback,  which  is  the 
number  of  times  each  video  Is  displayed  before  the 
next  frame  Is  shown,  determines  the  apparent  speed 
of  travel.  Free  choice  In  what  routes  may  be  taken 
I  s  obta  I  ned  by  filming  all  poss  ibl  e  paths  t  n  the 
area  as  well  as  all  possible  turns  through  all 
Intersections.  Whi le  It  might  first  appear  that 
this  would  be  a  time  consuming  and  expensive 
technology,  it  is  In  fact  relatively  efficient 
because  of  the  design  of  special  equipment  and 
procedures  for  doing  the  filming. 

Demonstrations  of  this  technology  have  been 
developed  for  building  Interiors  (MIT,  National 
Gallery  of  Art),  a  small  town  (Aspen,  Colorado),  an 
indusrlal  facility  (nuclear  power  plant),  a  weapon 
site,  and  San  Francisco  Harbor.  In  progress  Is  the 
production  of  a  prototype  video  mapping  library  of 
broader  scope  for  selected  areas  worldwide. 

To  provide  training  In  reading  and 
understanding  maps,  the  photograph-based  Surrogate 
Travel  is  linked  to  different  sorts  of  maps  of  the 
area.  In  effect,  the  viewer  can  travel  across  a 
map,  can  fly  into  it  getting  greater  and  greater 
detail  from  what  can  be  presented  by  standard  map 
symbology,  and  then  "fall  through"  the  map  to  see 
photographically  what  the  map  depicts.  I  r. 
addition,  the  viewer  can  switch  among  different 
types  of  maps  (e.g.,  topographic.  Infrared,  etc.) 
to  develop  an  understanding  of  how  different  map 
symbologies  and  representations  Interact. 

In  addition  to  ground  level  travel.  Including 
the  inside  and  outside  of  buildings,  aerial  flight 
experience  can  be  produced  and  used  for  flight 
training  and  airport  familiarization.  Similarly, 
other  forms  of  travel  experience,  such  as  anchorage 
piloting  end  low  level  nap  of  the  earth  flying,  are 
also  easily  accomodated. 

Surrogate  travel  can  also  be  used  to  provide 
training  on  routine  end  emergency  procedures. 
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physical  plant  maintenance,  safety,  security  as 
well  as  other  training  requirements  found  In  ships, 
military  and  industrial  facilities.  In  these 
appl Icatlons  blueprints,  floor  plans,  procedures, 
and  up-to-date  reference  materials  are  linked  with 
the  photography  of  the  site  to  provide  a  powerful 
and  easy-to-use  training  system. 

Electronic  Libraries 

Electronic  libraries.  Illustrated  In  Figure  3, 
In  the  form  of  Spatial  Data  Management  Systems 
(SOMS)  provide  students  and  Instructors  with  quick 
and  easy  access  to  an  assortment  of  multi-source 
and  multi-media  Information  <3).  Users  literally 
"fly  over"  Information  and  select  what  thev  want  by 
simply  pointing.  Spatlality  Is  use  to  group 
materials  into  lesson  plans  so  that  different 
Information  spaces  represent  course  concepts, 
additional  instruction,  and  assessment  procedures. 


i§:.  Illustration  o i  .in  Llc.'irrnio  l.iKi.irv. 


Stored  on  a  videodisc  are  tens  of  thousands  of 
frames  consisting  of  photographs,  diagrams,  charts, 
texts,  movies,  spoken  speeches,  music,  graphs,  etc. 
The  pages  can  be  organized,  reassembled,  segmented, 
and/or  duplicated  in  accordance  with  the  user’s 
need  and  growing  sophistication  wit",  the  subject 
matter.  The  pages  can  be  annotated,  highlighted, 
drawn-on,  underlined,  etc.,  at  the  user’s 
convenience  and  pleasure. 

For  the  Instructor,  the  SDMS  provides  ready 
access  to  a  wealth  of  material  which  might 
otherwise  be  unaccossible.  Instructors  can  access 
the  SDMS  to  create  their  own  Information  spaces 
(l.e.,  courses  or  lectures)  and  subsequently 
present  such  materials  to  large  audiences  In  single 
locations  via  large  screen  television  projection  or 
to  multiple  locations  through  cable  distribution 
systems. 

Students  can  Independently  use  the  SDMS  for 
self  paced  Instruction  by  either  working  through 
previously  designed  Information  spaces  or  by 
browsing  on  their  own.  When  students  and 
instructors  are  In  remote  locations,  offsite 
Instruction  Is  facilitated  by  linking  two  or  more 
SDMSs  together  using  rogu  I  ar  telephone  lines.  In 
this  manner,  a  student  or  Instructor  can  literally 
fly  the  other  to  a  topic  of  Interest,  sharing  at 
geographically  remote  sites  a  large  library  of 
information. 


The  same  video  materials  can  be  used  for 
hundreds  of  different  users.  The  only  thing  that 
must  be  changed  from  user  to  user  Is  the  magnetic 
storage  medium  which  serves  as  the  user's  private 
librarian  for  the  videodisc. 

The  U.S.  Marine  Corps  Education  Center  Is 
using  the  videodisc  and  SDMS  technology  to  create 
the  electronic  library  which  we  have  discussed. 
The  low  cost  replication  characteristics  of  the 
videodisc  make  this  medium  an  attractive 
alternative  for  the  distribution  of  previously 
archived  Information. 

I SSUES 

Development  and  use  of  training  applications 
employing  the  videodisc  such  as  those  discussed  In 
the  previous  section  raise  a  set  of  Issues  to  be 
considered  by  the  training  community.  A  few  of 
these  issues  are  briefly  discussed  below: 

1.  What  are  the  authoring  requirements  In  producing 
videodisc  based  training  systems?  Probably  the 
most  Important  requirement  Is  the  need  to  work 
freely  with  the  capability  of  storing  and  almost 
Instantaneously  accessing  over  50,000  frames  of 
Information.  Our  experience  Is  that  many 
videodisc  first-timers  make  only  incremental  use 
of  the  technology  because  they  do  not  take 
advantage  of  Its  random  access  characteristics. 
Missing  from  the  videodisc  milieu  are 
Inexpensive  authoring  systems  that  would  enable 
authors  to  put  together  and  try  out  new 
videodisc  program  concepts.  Bunderson  and 
Campbell  (1)  have  prepared  a  more  complete 
discussion  of  this  Issue. 

2.  What  Is  the  role  of  digitized  sound  and 
synthesized  speech  In  videodisc  appl Icatlons? 
Currently,  several  groups  are  experimenting  with 
a  number  of  means  tc  overcome  the  lack  of  still 
sound  I n  vi deodi sc  systems.  Future  vl deod I  sc 
systems  will  almost  certainly  Incorporate  still 
sound  capabilities.  However,  synthesized  speech 
may  have  an  Important  place  in  videodisc 
applications  given  the  rapid,  random  access 
capabilities  of  videodiscs.  For  example,  in  the 
Surrogate  Travel  application  discussed  earlier, 
the  controlling  computer  also  generates 
narrative  descriptions  that  accompany  the 
viewer's  travel.  Speech  synthesis  is  used  as 
the  audio  channel  because  It  I  s  not  reasonab I e 
to  prestore  all  things  that  could  be  said  for 
all  possible  routes  that  the  viewer  might  take. 

3.  How  does  videodisc  technology  affect  the 
accessibility  of  training  devices?  A  lot.  With 
the  videodisc,  the  training  device  designer  gets 
a  comparatively  low  cost  (potentially  less  than 
J1,000)  system  that  brings  together  an  array  of 
media  (slides,  text,  graphs,  movies,  sound)  In  a 
compact  and  robust  form.  In  some  Instances, 
very  expensive  computer  graphics  based  systems 
may  be  replaced  by  Inexpensive  videodisc 
players. 

4.  How  "smart"  a  videodisc  training  device  Is 
needed?  Smart,  yes,  but  not  br  I  Pliant.  The 
applications  discussed  earlier  ell  use  videodisc 
players  controlled  by  small,  inexpensive 
microcomputers.  Future  videodisc  players  may 
Incorporate  sufficiently  capable  mlcroprocesor 
systems  to  allow  the  necessary  functions  to  be 


147 


completely  on  board.  While  It  Is  true  that  the 
players  available  today  include  a  small 
microprocessor  which  can  support  branched 
Instruction--  usage  In  this  mode  Is  far  removed 
from  the  real  potential  of  the  videodisc  as  a 
tral  nlng  device. 

5.  How  can  training  take  advantage  of  the  "super 
realism"  offered  by  videodiscs?  In  the 
Surrogate  Travel  and  low  cost  trainer 
applications,  the  viewer  sees  the  real  place 
(e.g.,  a  real  tank  and.  If  appropriate,  a  real 
explosion).  This  Is  all  made  possible  by 
storing  a  fasclmlle  version  of  events  and 
materials  on  the  video  disc.  Thus,  Instead  of 
seeing  a  simplification  (although  at  times  this 
may  be  beneficial),  the  user  Interacts  with  the 
object  or  process  as  It  wl  I  I  be  encountered  In 
the  work  place. 

6.  Are  videodisc  training  devices  Inherently  "fun"? 
Stated  In  another  form,  since  television  Is  fun, 
will  videodisc  training  devices  find  greater 
acceptance?  Without  attempting  a  full  analysis 
we  state  our  experience:  Users  enjoy  working 
with  these  systems.  We  cannot  say  If  this  Is 
because  the  systems  are  well  designed,  because 
they  Include  color,  sound,  and  motion,  because 
they  are  a  familiar  medium  (television),  or 
because  of  something  else.  Nonetheless,  the 
overwhelming  Impression  we  and  others  have 
gained  from  working  with  videodisc  systems  Is 
that  users  like  them. 

The  list  of  Issues  above  Is  In  no  way 
complete.  What  we  have  attempted  to  do  Is  alert 
the  reader  to  some  of  the  questions  which  have 
arisen  through  our  own  experiences.  We  feel  that 
the  area  of  videodisc  based  training  devices  Is 
extremely  promising  and  deserves  considerable 
attention  by  the  training  community. 
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ABSTRACT 


Performance  errors  committed  during  the  acquisition  of  a 
carrier  landing  task  In  the  simulator  resulted  In  the  occurrence 
of  a  freeze.  Pilot  subjects  exposed  to  the  freeze  developed 
control  strategies  which  were  distinguishable  from  those  of  a 
no-freeze  control  group  In  terms  of  throttle,  rudder,  aileron, 
and  elevator  activity.  Neither  rate  of  learning  nor  level  of 
performance  was  affected.  Use  of  the  freeze,  however,  was 
reported  as  Initially  "frustrating"  and  as  adding  to  the  overall 
difficulty  of  acquiring  the  task.  With  certain  applications,  the 
freeze  appears  to  be  aversive,  altering  pilot  motivation  for 
learning  the  task.  Methodologically,  the  study's  use  of  a  probe 
trial  technique  represents  a  departure  from  the  traditional 
transfer  of  training  methodology  for  the  assessment  of  alterna¬ 


tive  Instructional  treatments. 

INTRODUCTION 

Error-free  and  trial -and-error  approaches 
represent  extreme  positions  with  respect  to 
the  treatment  of  errors  during  learning. 
(6,13,14)  The  traditional  approach  to  flying 
training  lies  somewhere  between  these 
extremes.  Not  until  the  recent  widespread  use 
of  flight  simulators  did  the  Instructor  even 
have  a  choice  with  respect  to  how  errors  could 
be  dealt  with  instructlonally.  Now  that 
instructional  options  are  available,  one 
question  that  arises  Is  the  extent  to  which 
errors  should  be  permitted  to  freely  occur 
during  training  and  the  extent  to  which  errors 
actually  affect  subsequent  performance.  (13) 

The  specific  role  of  errors  In  skill 
acquisition  is  unknovm.  Holding  has  expressed 
the  concern  that  frequently  committed  erro-s 
may  be  learned,  or  If  not  actually  learned  in 
the  sense  of  being  committed  more  frequently, 
that  they  mav  become  embedded  In  the  student's 
response  repertoire.  (6)  This  outcome  appears 
possible  In  the  case  nf  the  carrier  landing 
task  studied  here  since  errors  often  seem  to 
be  reinforced.  For  example,  a  marginal 
approach,  although  being  severely  criticized 
bv  the  Landing  Signal  Officer  (LSO),  can 
result  In  a  safe  touchdown.  Although  Navy 
pHots  are  consciously  and  explicitly 
concerned  with  technique  In  making  carrier 
approaches,  successful  completion  of  a 
dangerous  task  by  any  means  is,  In  a  strict 
sense,  reinforcing.  Undoubtedly,  the  natural 


consequences  of  errors  has  a  powerful 
self-correcting  Influence  In  perceptual  motor 
learning,  but  the  natural  consequences  of 
errors  In  carrier  approaches  may  not  be 
sufficiently  negative  with  the  required 

frequency  to  give  full  force  to  this  effect. 

The  present  study  sought  to  address  those 
Instructional  approaches  to  teaching  the 
carrier  approach  task  that  are  possible  given 
the  Instructional  environment  of  the  flight 
simulator.  The  carrier  landing  task  was 
considered  ideal  for  the  present  study. 

Control  behavior  is  strictly  constrained  by 
the  demands  of  the  task,  and  errors  can  be 
clearly  specified.  Navy  pilots  and  instructors 
almost  unanimously  agree  that  error  detection 
and  correction  are  fundamental  to  safe  and 
consistent  carrier  landings. 

Specifically,  the  study  addresses  the  use 
of  the  simulator's  FREEZE  feature  to  Interrupt 
an  otherwise  continuous  performance  whenever 
an  error  Is  aetected.  The  simulator 
environment  also  makes  it  possible  following 
an  error  to  either  allow  the  errorful 

performance  to  continue  or  to  make  some 
correction  (e.g.,  putting  the  aircraft  back  on 
course)  prior  to  continuing  the  task.  A 
potential  Instructional  advantage  of  using  the 
FREEZE  Is  that  of  allowing  the  student  to 
attend  to  Instructional  feedback  without  at 
the  same  time  having  to  continue  to  fly  the 
aircraft.  A  potentially  negative  aspect, 
however,  is  that  the  FREEZE  may  function  like 
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a  T1me  Out  (TO),  a  behavior  modification 
method  frequently  used  to  suppress 
Inappropriate  or  undesirable  behavior. 

An  impo-tant  methodological  Issue  Is  also 
addressed  by  the  present  study,  that  of  how  to 
assess  the  differential  effectiveness  of 
alternative  experimental  treatments  (In  this 
case,  instructional  conditions)  upon 
subsequent  ability  to  perform  the  criterion 
task:  the  familiar  transfer  of  training 
problem.  One  important  issue  In  transfer  of 

training  research  <s  related  to  the  selection 
of  an  appropriate  training  period.  Training 
to  a  proficiency  criterion  has  often  been 
used,  but  in  a  study  of  differential  transfer, 
the  average  time  for  various  groups  to  attain 
proficiency  almost  Invariab'ly  differs.  Thus 
training  time  tends  to  be  confounded  with  the 
experimental  effects  of  interest.  Fixed 
training  times  resolve  that  problem  but 
selection  of  an  appropriate  period  can  be 
critical,  and  necessarily  relies  heavily  on 

the  ludgement  and  experience  of  the 
experimenter.  Training  times  could  be  too 

short  to  allow  differences  to  emerge. 
Alternatively,  they  could  be  so  long  that 

worthwhile  training  deferences  are  worked  out 
by  subjects  attaining  a  high  level  of 
proficiency  with  even  the  poorest  training 
conditions.  Thus  training  times  should  be 
extended  Into,  but  not  beyond,  that  period  in 
training  that  shows  worthwhile  learning 
differences  between  instructional  methods. 

Pre-experlmenta1  work  could  ascertain  the 
most  appropriate  training  period  but  it  would 
require  expenditure  of  a  large  portion  of  the 
experimental  resources  to  obtain  a  reliable 
answer.  Furthermore,  In  a  study  of  more  than 
two  training  conditions,  the  selected  training 
time  may  be  appropriate  for  only  some  of  the 
comparisons.  A  range  of  times  could  be  used, 
but  to  do  so  would  only  reduce  the  power  of 
the  experiment  H.e.,  its  capability  to  reveal 
differences  between  conditions)  to  the  extent 
that  some  of  the  selected  training  periods  are 
1  napproprlate. 

The  alternative  addressed  in  the  present 
study  Is  that  of  a  "probe"  technique.  A  probe 
technique  in  which  learning  trials  are 
interspersed  with  test  (criterion)  trials 
could  avoid  the  problems  discussed  above. 
This  technique,  which  appears  to  have  been 
used  only  once  In  applied  transfer  of  training 
research  (15),  might  effectively  map  the 
course  of  learning  and  thus  allow  an  estimate 
of  the  optimun  training  period  for  each 
instructional  method.  Smith  et  al.  (15)  used 
a  single  trial  probe  strategy  in  which  they 
alternated  training  and  probe  trials;  a 
strategy  that  was  probably  not  optimum. 
Presumably,  an  experimental  session  should  be 
weighted  heavily  with  training  versus  probe 
trials  to  ilm't  dilution  of  the  training 
effects.  Nevertheless,  probes  should  be 
frequent  enough  to  ensure  that  critical 
differences  are  not  missed,  and  sufficient 
data  would  be  required  at  each  probe  to 
achieve  worthwhile  stability.  Probe 
methodology  would  seem  to  offer  distinct 


advantages  for  the  Initial  Investigation  of  a 
novel  training  method.  However,  for 

evaluating  Its  savings  in  relation  to  a 
standard  instructional  paradigm,  the 
traditional  transfer  of  training  paradigm 
would  still  be  preferred. 

Suimary  of  Primary  Objectives 

1.  To  assess  the  relative 

effectiveness  of  three  Instructional  methods 
differing  in  the  degree  to  which  each  alters 
the  instructional  environment  following  an 
error . 

2.  To  determine  the  effects  of 
displacement -only  versus  displacement- 
displacement-plus-rate  error  criteria  on 
acquisition  of  the  approach  to  landing  task. 

3.  To  assess  the  extent  to  which 
criterion  performance  sampled  periodically  on 
probe  trials  is  sensitive  to  what  is  being 
learned  on  training  trials. 

METHOD 

Subjects 

Five  groups  of  experienced  Air  Force 
pilots  were  taught  carrier  landings  in  a 
flight  simulator  at  the  Naval  Training  and 
Equipment  Center  under  a  control  or  one  of 
four  experimental  training  conditions.  Pilot 
subjects  averaged  35  years  of  age,  had 
approximately  2400  flying  hours,  300  simulator 
hours,  and  approximately  20  hours  in  their 
assigned  aircraft  (F-4  or  F-16)  in  the  30  days 
preceeding  the  study. 

Apparatus 

The  Visual  Technology  Research  Simulator 
(VTRS)  consists  of  a  fully  instrumented  T-2C 
Navy  jet  trainer  cockpit,  a  six 
degree-of -freedom  synergistic  motion  platform, 
a  32-element  G-seat,  a  wide  angle  visual 
system  that  can  project  both  computer 
generated  and  model  board  Images,  and  an 
Experimenter/Operator  Control  Station.  (5) 
The  motion  system,  G-seat  and  model  board  were 
not  used  In  this  experiment. 

Visual  System 

The  background  subtended  59°  above 
to  39°  below  the  pilots'  eye  level,  and 
80°  to  either  side  of  the  cockpit.  The 
carrier  Image,  which  was  a  representation  of 
the  Forrestal  (CVA  59)  was  generated  by 
computer  and  projected  onto  the  background 
through  a  1025-line  video  system.  A  carrier 
wake  and  FLOLS  were  also  generated  by  this 
method.  Both  day  and  night  carrier  Images 
could  be  displayed. 

Average  delay  between  control  inputs 
and  generation  of  the  corresponding  visual 
scene  was  approximately  116  msec.  Calculation 
of  new  aircraft  coordinates  required  50  msec 
while  calculation  of  the  coordinates  for  the 
visual  scene  corresponding  to  the  viewpoint 
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from  the  new  aircraft  coordinates  required 
approximately  50  msec.  Generation  of  the  new 
scene  required  16  msec.  An  updated  visual 
scene  was  displayed  every  31  msec. 

The  sky  brightness  for  the  day  scene  was 
0.85  fL  (foot-lambert)  and  the  seascape 
brightness  was  0.6  fL.  The  brightness  area  of 
the  day  carrier  was  4.0  fL.  Except  fo^  the 
horizon,  there  were  no  features  represented  In 
either  the  sky  or  sea.  The  night  background 
luminance  was  0.04  fL  and  the  horizon  and 
seascape  were  not  visible.  The  night  carrier 
appeared  as  light  points  of  0.8  fL  brightness 
outlining  the  landing  deck  and  other  features. 

Fresnel  Lens  Optical  Landing  System  (FLOLS) 

In  contrast  to  a  carrier  FLOLS, 
which  Is  generated  by  Incandescent  lights,  and 
can  therefore  be  much  brighter  than  other 
parts  of  the  carrier,  the  simulated  FLOLS  was 
generated  by  the  same  system  as  the  carrier 
Image.  It  was,  therefore,  only  as  bright  as 
the  brightest  areas  of  the  ship  (e.a.,  the 
white  lines  on  the  landing  deck).  To 
compensate  for  Its  lower  relative  brightness, 
the  FLOLS  was  enlarged  by  a  factor  of  three 
when  the  distance  behind  the  ramp  was  greater 
than  2250  ft.  From  2250  ft  the  size  of  the 
FLOLS  was  M nearly  reduced  until  It  attained 
Its  normal  size  at  750  feet.  The  FLOLS  was 
centered  414  ft  down  the  landing  deck  and  61 
ft  to  the  left  of  the  centerline.  It  was  set 
at  a  nominal  3.5°  glideslope  and  with  a 
lateral  viewing  wedge  of  52°. 

Simulator  Configuration 

The  simulator  was  Initialized  with  the 
aircraft  at  9000  ft  from  the  ramp,  on  the 
glideslope  and  centerline,  and  In  the  approach 
attitude  and  configuration  (hook  and  wheels 
down,  speed  brake  out,  ’5  units  AOA,  and  power 
at  8310 .  The  T-2C  Is  normally  landed  with 
full  f’aps,  hut  flaps  were  set  at  half 
extension  for  this  experiment  to  more  closely 
simulate  approach  speeds  of  typical  fleet 
aircraft.  Fuel  was  set  at  1320  lbs  to  give 
10,000  lbs  gross  weight.  A  landing  trial  was 
flown  from  the  initial  condition  to  wire 
arrestment  or.  In  the  case  of  a  bolter,  to 
1000  ft  past  the  carrier.  The  carrier  was  set 
on  a  heading  of  360°  at  5  knots. 
Envlromental  wind  was  set  at  349.°  with  a 
velocity  of  20.1  knots.  This  combination  of 
carrier  speed  and  environment  wind  produced  a 
relative  wind  component  of  25  knots  directly 
down  the  landing  deck.  Turbulence  was  used  to 
increase  the  difficulty  of  the  task.  The 
turbulence  model  buffetted  the  simulator  with 
a  random  forcing  fuctlon. 

PROCEDURE 

Two  subjects  arrived  at  the  simulation 
facility  each  day,  Monday  through  Thursday, 
during  the  experiment.  They  viewed  a  video 
tape  on  carrier  landings  which  described  the 
FLOLS  and  carrier  landings.  They  were  then 
given  detailed  Instructions  by  a  Navy  L50  on 
carrier  landing  techniques.  This 


instructional  period  lasted  approximately  45 
minutes.  Where  convenient,  subjects  were 
given  their  preliminary  instruction  In  pairs, 
but  the  remaining  experimental  work  was 
undertaken  with  only  one  subject  In  attendance 
except  that  subjects  were  occasionally 
permitted  to  monitor  the  performance  of  others 
from  outside  the  simulator  if  they  had 

entirely  completed  their  experimental  work. 
Subjects  were  assigned  to  training  conditions 
as  they  arrived  at  the  simulator  facility  In 
accordance  to  a  predetermined  sequence. 

Training  Conditions 

Two  experimental  training  procudures  In 
addition  to  a  control  (CONVENTIONAL)  procedure 
were  used  In  the  experiment.  For  both 

experimental  procedures,  the  subject  pilots 
were  frozen  during  the  approach  If  their 

vertical  deviations  from  the  glideslope  broke 
specific  criteria.  Under  one  procedure,  known 
as  FREEZE/RESET,  when  the  pilots  were  frozen 
they  were  Instructed  on  how  they  had  Incurred 
their  vertical  error  and  were  then  reset  to 

the  glideslope  with  the  simulator  in  its 
optimum  approach  attitude.  Longitudinal 
distance  from  the  carrier  and  lateral  distance 
from  the  extended  centerline  of  the  landing 
deck  were  not  changed.  Pilots  continued  their 
approach  from  the  reset  position.  Under  the 
other  experimental  procedure,  known  as 
FREEZE/FLYOUT,  pilots  were  instructed  on  how 
they  had  incurred  their  vertical  error  and  how 
to  correct  it  once  they  were  released.  They 
then  continued  their  approach  from  the 

position  and  attitude  in  which  they  had  been 
frozen. 

Two  different  experimental  training 
conditions  were  derived  from  each  FREEZE 
procedure  by  applying  two  different  freeze 
criteria.  The  first  froze  the  system  If 
|®il'!®ct  #*iere 

9^  »  angular  displacement  of  the 

aircraft  from  the  3.5  degree 
glideslope, 

9C  =  0.5625  -  (.3125  x  10‘4),  and  where 

r  =  range  in  feet  from  the  carrier  ramp 

This  algorithm  linearly  increased  the 
criterion  in  meatball  units  from  1.0  at  6000 
ft  from  the  ramp  to  1.5  at  the  ramp.  Freezes 
did  not  occur  beyond  6003  ft  or  past  the  ramp. 

The  second  criterion  would  result  in  a 
freeze  If  vertical  deviation  from  the  3.5° 
glideslope,  or  decent  rate  error,  or  some 
combination  of  the  two  was  excessive.  Freezes 
would  occur  If  JH, |  >  0C  for 

M-j  *  9-j  +  0.5625 

9^  ■  angular  rate  of  displacement  In 

degrees/second  from  the  glideslope 

9C  =  0.405  -  (.49  x  10-4) (r  +  rfc) 
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*  524  feet,  the  distance  of  the 
carrier  from  the  FLOLS 
origin. 

Th*s  algorithm  established  a  criterion  that 
was  a  weighted  sum  of  the  previously  described 
displacement  criterion  and  a  descent  rate 
error  limit  that  decreased  linearly  from  600 
fpm  at  6000  ft  from  the  ramp  to  200  fpm  at  the 
ramp. 

None  of  the  experimental  training 
conditions  permitted  a  freeze  within  10 
seconds  of  restarting  the  approach  after  a 
freeze.  In  adddltion,  a  freeze  would  not 
occur  If,  at  the  end  of  this  10-second  period, 
the  subject  was  outside  of  the  performance 
criterion  but  was  descreasing  his  error. 

In  a  fifth  training  condition,  designated 
as  CONVENTIONAL,  subjects  were  not  frozen 
during  the  approach  but  were  given  their  error 
feedback,  equivalent  In  nature  to  that  given 
the  FREEZE/FLYOUT  group,  during  and  at  the  end 
of  each  trial. 

After  preliminary  instruction,  subjects 
were  familiarized  with  the  controls  of  the 
simulator.  They  were  then  given  a  brief 
flight  of  approximately  two  minutes  before 
they  commenced  their  carrier  landing 
training.  The  training  sequence  consisted  of 
24  approaches  to  the  day  carrier  on  the 
afternoon  of  their  first  day  at  the  simulator 
facility,  and  24  approaches  to  the  night 
carrier  on  the  morning  of  their  second  day. 
The  two  24-trlal  blocks  were  divided  into 
6-triai  sub-blocks,  the  first  4  trials  of 
which  were  flown  under  the  appropriate 
training  condition.  The  iast  2  trials  of  each 
sub-block  were  used  as  probe  trials  to  assess 
the  progress  of  learning,  and  were  flown  under 
the  control  condition.  The  LSO  gave  no 
instructions  during  or  following  probe 
trials.  Subjects  were  given  a  IG-minute  rest 
after  the  twelth  tria1  of  each  24-trlal  block. 

Performance  Measurement 

Parameters  of  aircraft  position  and 
attitude  were  sampled  at  30  Hz  and  used  to 
derive  altitude  and  lineup  error  scores  from 
the  desired  approach  path,  and  deviations  from 
desired  AOA  (15  units).  Root  Mean  Square 
(RMS)  error,  mean  algebraic  error  and 
variability  around  that  mean  were  calculated 
for  these  three  dependent  variables  or  four 
equal  segnents  of  the  final  6000  ft  of  the 
approach. 

Glideslop.  and  lineup  errors  at  4500, 
3000,  2000,  1000,  and  0  ft  from  the  ramp  were 
used  to  derive  means  and  standard  deviations 
at  these  five  points  in  the  approach. 
Distance  from  the  deck,  distance  from  the 
centerline,  and  descent  rate  were  measured  at 
touchdown,  and  the  Landing  Performance  Score 
(LPS)  was  calculated.  (3)  The  LPS  Is  a  score 
assigned  to  each  pass,  ranging  from  1.0 
(technique  wave  off)  to  6.0  (#3  wire  trap). 

Lateral  stick,  longitudinal  stick,  rudder 


pedal,  and  throttle  positions  were  sampled  at 
30  Hz.  The  distance  of  control  movement  from 
one  sampling  point  to  the  next  was  accumulated 
over  one  second  periods  and  averaged  over  four 
equal  segments  of  the  final  6000  ft  of  the 
approach. 

RESULTS 

Day  Versus  Night  Performance 

The  data  showed  little  evidence  for 
significant  improvements  in  student  pilot 
performances  during  the  night  carrier 
approaches.  Isolated  effects  during  the  night 
trials  included  a  significant  reduction  In 
aileron  movement  in  the  area  of  6000-4000  ft 
to  the  ramp;  a  reduction  in  glideslope 
variability  from  1000  ft  to  the  ramp;  and  a 
reduction  In  throttle  movement  from  3000-1000 
ft  to  the  ramp.  Since  performance  appears  to 
have  stabilized  during  the  day  carrier  trials, 
results  are  discussed  for  the  day  trials  only 
except  as  where  noted  otherwise  (specifically, 
the  correlational  results  on  the  probe  vs. 
training  trial  performances  and  the 
discriminant  function  analysis). 

Use  of  Displacement  Versus  Displacement - 
Plus-kate  Error  Criteria 

Neither  rate  of  learning  nor  level  of 
performance  was  significantly  affected  by  the 
particular  error  criterion  in  effect;  that  is 
to  say,  whether  the  freeze  was  occasioned  by 
deviations  in  vertical  displacement  or 
deviations  in  vertical  displacement  from 
glldepath  plus  rate  of  descent.  Although  Kaul 
et  al.  (8)  found  significant  differences  for 
experienced  Navy  pilots  using  displacement- 
plus-rate  information  displayed  on  the  FLOLS, 
the  Kaul  study  and  the  present  study  were 
different  in  several  respects.  (8)  First,  the 
displacement-plus-rate  information  in  the  Kaul 
study  was  presented  to  the  pilot  continuously 
via  a  modified  FLOLS  display.  In  the  present 
study,  the  conventional  FLOLS  was  displayed 
regardless  of  which  error  criterion  was  In 
effect.  Second,  the  present  study  contained 
no  condition  which  exactly  paralled  the 
displacement-plus-rate  condition  in  the  Kaul 
study.  In  the  present  study,  there  was  no 
condition  where  subjects  performed  under  the 
CONVENTIONAL  (i.e.,  no  freeze)  criterion  with 
feedback  provided  in  terms  of  displacement 
plus  rate.  Although  the  subject  population 
(experienced,  carrier  qualified  Navy  pilots 
versus  experienced,  but  not  carrier  qualified 
Air  Force  pilots)  also  differed,  the 
similarity  of  the  glideslope  maintenance 
portion  of  the  task  was  highly  similar  for 
both  groups  of  pilots. 

Relationship  Between  Performance  on  Training 
Trials  and  Subsequent  Probe  Trials 

Pearson  product  moment  correlation 
coefficients  were  computed  between  measures  of 
performance  on  training  trials  and  subsequent 
probe  trials  for  subjects  in  both  the 
FREEZE/FLYOUT  and  CONVENTIONAL  conditions. 
Data  were  combined  across  both  day  and  night 


landing  trials.  No  correlations  are  reported 
for  the  FREEZE/RESET  condition  since  use  of 
the  RESET  feature  precludes  use  of  variability 
data.  Correlational  data  are  not  presented 
here  due  to  limitations  on  published 
presentation  length.  Nevertheless,  the 
following  observations  can  be  made  from  these 
data. 

1.  During  the  approach  portion  of  the 
task,  a  high  degree  of  consistency  was 
observed  between  pilot  control  movements  (l.e. 
pilot  Inputs  to  aileron,  pedal,  and  elevator) 
on  training  trials  and  subsequent  probe 
trials.  This  was  true  to  a  lesser  extent  for 
throttle  control.  This  suggests  that  for 
subjects  within  a  given  Instructional 
condition  the  same  or  similar  control 
strategies  used  during  training  were  also  used 
during  probe  trials. 

2.  Even  though  control  Inputs  appear  to 

have  been  consistent  across  training  and  probe 
trials,  the  consistency  of  training  and  probe 
trial  touchdown  performances  differed  as  a 
function  of  Instructional  method.  The 
correlation  between  the  wire  caught  on  probe 
and  training  trials  was  highly  correlated  for 
the  CONVENTIONAL  group  (r«0.60)  but  not  for 
the  FREEZE/FLYOUT  group  (r«0.17).  Likewise, 
pitch,  vertical  velocity  at  touchdown,  and 
angle  of  attack  at  touchdown  were  significantly 
correlated  for  the  CONVENTIONAL  group 
(correlations  of  0.64,  0.78,  and  0.53, 

respectively)  but  not  for  the  FREEZE/FLYOtfT 
group  (correlations  of  -0.11,  0.32,  and 

-0.10, respectively).  The  extent  to  which 
performances  acquired  during  training 
transfered  to  the  criterion  conditions  of  the 
probe  trial  is  also  expressed  In  terms  of  the 
correlations  between  Landing  Performance 
Scores  (LPS)  on  training  and  probe  trials.  For 
the  Conventional  group  this  correlation  was 
0.35;  for  the  Freeze/Flyout  group,  0.18. 

Did  Learning  Occur 

The  data  showed  a  small  but  statistically 
significant  reduction  In  freezes  (errors) 
across  successive  blocks  of  probe  trials, 
F(3,60)-3.62,  p=0.018,  indicating  that 

learning  did  occur.  Comparisons  by 
Instructional  condition,  however,  revealed  no 
slgnflcant  differences  due  to  group 
assignment,  F(4,20)=0.94,  p=0.46.  The 

learning  effects  are  small  when  considered 
from  an  operational  standpoint.  The 
opportunity  for  measured  errors  to  occur  war; 
bounded  by  the  10-second  limitation  Imposed  on 
the  freeze  as  well  as  by  not  freezing  when  a 
rate  error  was  In  effect  but  the  pilot  was 
correcting.  Neither  can  the  effect  of 
subjects'  experience  be  overlooked.  Based  on 
their  experience  with  constant  airspeed,  angle 
of  attack  approaches,  one  might  expect  that 
the  carrier  task  differed  only  In  close  to  the 
ship  (e.g.,  3000  ft  to  the  ramp).  This  Is 
supported  by  the  data  which  show  significant 
practice  effects  occurring  most  often  In  the 
1000  ft  to  the  ramp  segnent  of  the  task.  Only 
In  the  case  of  aileron  movement  did 
significant  learning  effects  occur 


consistently  In  all  segments  of  the  task  from 
6000  ft  to  the  ramp. 

Process  Versus  Product  Measures  of  Performance 

A  step-wise  discriminant  function  analysis 
(BMPD7M)  was  performed  on  the  performance 
measures.  As  the  data  set  did  not  conform  to 
rules  normally  required  for  a  discriminant 
analysis  (e.g.,  the  size  of  the  smallest  group 
should  exceed  the  number  of  variables),  its 
use  here  should  be  considered  as  purely 
exploratory.  (16)  The  cumulative  proportion 
of  total  dispersion  accounted  for  by  the 
discriminant  function  was  48*,  and  the 

variables  that  contributed  most  to  separation 
of  the  groups  were  those  of  control  movements 
(pedal  movement  from  4500-3000  feet  from  the 
ramp,  throttle  movement  from  1500  feet  from 
the  ramp  to  touchdown,  and  aileron  movement 
from  4500  to  3000  feet  from  the  ramp).  The 
discriminant  function  tended  to  distinguish 
the  control  (CONVENTIONAL)  group  from  the 

experimental  groups,  and  in  all  cases  the 

trends  In  the  data  were  towards  smoother 
control  Inputs  for  pilots  in  the  CONVENTIONAL 
(no  freeze)  condition. 

The  results  of  the  discriminant  analysis 
agree  with  those  of  the  correlational  analysis 
In  so  much  as  they  point  to  pilot  control 
responses  as  being  the  dimension  most 
significantly  affected  by  variations  In 
Instructional  approach.  It  appears  from  the 
data  that  those  pilots  exposed  to  the  freeze 
adopted  control  strategies  that  were  different 
than  those  adopted  by  pilots  In  the 
CONVENTIONAL  Instructional  condition.  It  is 
hypothesized  that  these  differences  were 
acquired  during  training  as  part  of  the  task 
of  "coming-off -freeze."  The  probe  trial 
procedure  was  sensitive  to  the  transfer  of 
variations  in  control  Inputs  from  training 
trials  to  probe  trials  even  though  customary 
training  outcome  measures  of  performances  on 
probe  trials  were  not  found  to  differ  as  a 
function  of  Instructional  condition. 

Questionnaire  Responses 

Following  participation  In  the  study,  each 
pilot  subject  in  the  FREEZE/FLYOUT  and 
FREEZE/RESET  groups  completed  a  questionnaire 
(see  Appendix).  The  results  are  stumor  1  zed 
below. 

On  the  General  Role  of  Errors  In  Training 

Pilots  generally  disagreed  that 
"errors  served  little  purpose"  as  well  as  with 
the  notion  that  "students  may  actually  learn 
the  error-  hey  commit"  (Item  12).  Pilots 
also  disagreed  with  the  contention  that 
"instructional  methods  that  allow  errors  to 
occur  are  Inefficient"  (Item  14).  Instead, 
pilots  in  the  study  pointed  to  error 
recognition  as  a  basis  for  the  development  of 
correct  performance  (Item  18).  Errors  were 
seen  as  helping  the  student  to  focus  on  the 
critical  elements  of  task  performance  (Item 
13),  as  well  as  exposing  the  student  to 
out-of -tolerance  situations  which  may  under 
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later  conditions  result  from  factors  such  as 
adverse  weather,  vlslblllty/cel ling 

limitations,  etc.  (Item  15).  On  the  Issue  of 
whether  correct  performance  Is  best  thought  of 
as  resulting  from  a  process  of  eliminating 
errors  or  from  a  process  of  shaping  desired 
responses,  pilots  were  undecided  (Item  17). 

On  the  Instructional  Use  of  the  Freeze 
Feature 

Pilots  agreed  that  It  was 
significantly  easier  to  attend  to  the  LSO's 
feedback  while  the  simulator  was  frozen  than 
while  trying  to  listen  and  fly  the  aircraft 
simultaneously  (Item  3).  Pilots  also  agreed 
that  use  of  the  the  freeze  aided  In  the 
development  of  error  recognition  (Item  5). 
Pilots,  however,  were  undecided  as  to  whether 
the  use  of  the  freeze  might  be  used  to 
significantly  decrease  the  overall  training 
time  required  to  1ea<-n  the  landing  task.  On 

the  negative  side,  pilots  Indicated  that  the 
occurrence  of  the  freeze  early  In  training 
was  "frustrating"  (Item  8).  In  fact,  pilots 
In  the  FREEZE/RESET  condition  Indicated  that 
they  were  more  motivated  by  "trying  to  avoid 
the  freeze  than  by  trying  to  fly  the  task 

correctly"  (Item  7).  Reponses  of  pilots  In 

the  FREEZE/FLYOUT  condition  to  the  same  Item 
did  not  reflect  this  Implied  aversive  aspect. 
Regardless  of  the  freeze  condition  to  which 
subjects  were  assigned,  all  Indicated  that 
regaining  control  of  the  simulator  following  a 
freeze  significantly  added  to  the  difficulty 
of  the  flying  task  (Item  21,  and  that  the 

difficulty  Increased  the  closer  the  freeze 
occurred  to  actual  touchdown  (Item  2). 

In  general  then  the  questionnaire  data 
indicated  that  pilots  perceived  errors  as 
contributing  positively  to  training;  that  the 
oresent  use  of  the  freeze  was  In  some 
Instances  aversive  and  that  It  served  to  add 
to  the  difficulty  of  learning  the  task  In  the 
simulator  despite  the  fact  that  the  freeze 
made  It  easier  to  attend  to  the  feedback  from 
the  LSO  instructor.  So  far  as  being  able  to 
ootentlally  reduce  the  time  needed  to  learn 
the  task,  pilots  perceived  the  present 
applications  of  the  freeze  to  have  little 
value  In  this  regard. 

SUMMARY  AND  CONCLUSIONS 

The  view  of  the  freeze  as  an  opportunity 
to  review  past  events  and  to  plan  for  the 
future  Is  consistent  with  an  information 
processing  approach  to  learning.  (9)  Under 
this  view,  competing  Information  processing 
activities  can  Interfere  and  retard 
acquisition  of  perceptual -motor  skills.  There 
was  no  evidence  that:  this  w*«  the  r»«»  the 
present  study,  even  though  pilots  Indicated 
that  It  was  easier  to  attend  to  the  LSO's 
instructional  feedback  while  frozen  than  to  do 
so  while  continuing  to  perform  the  task 
simultaneously.  Subjects  In  the  present  study 
were,  however,  experienced  Air  Force  pilots 
even  though  naive  with  respect  to  the  carrier 
landing  task.  Those  studies  which  suggest 
that  experienced  pilots  are  better  able  to 


time  share  than  less  experienced  pilots  or 
non-pilots  would  suggest  that  differences 
might  have  emerged  had  subjects  come  from  a 
less  experienced  subject  population. 

The  specific  relevance  of  the  Information 
processing  notion  to  flight  Instruction  has 
not  been  fully  established.  Other 
psychological  processes  m^y  play  a  role  In 
mediating  the  effect  of  the  freeze.  For 
example,  the  freeze  could  well  be  aversive, 
functioning  in  the  same  manner  as  a  "time-out 
from  positive  reinforcement"  (usually  referred 
to  simply  as  T.O.)  has  been  shown  to  function 
In  the  control  of  undesirable  or  inappropriate 
behavior.  (2)  Pilot  comnents  In  the  present 
study  support  the  view  that  the  freeze  may  be 
viewed  as  aversive  and  that  some  applications 
of  the  freeze  (e.g.,  the  FREEZE/RESET)  may 
even  significantly  alter  the  nature  of  the 
student's  motivation  to  learn  the  task.  All 
agreed  that  use  of  the  freeze  was 
"frustrating"  early  In  training.  That  one  or 
more  of  these  processes  can  affect  the 
acquisition  of  a  psychomotor  task  Is  also 
Indicated  by  the  work  of  Payne  and  his 
associates.  (10,  11,12) 

Furthermore,  there  can  be  no  certainty 
that  the  freeze  effect  will  facilitate 
learning.  The  Interruption  caused  by  the 
freeze  may  disrupt  the  Integrity  of  the  task 
In  a  manner  reminiscent  of  part-task 
learning.  It  was  clear  In  the  present  study 
that  the  use  of  the  freeze  produced 
differences  In  how  pilots'  performed  the 
landing  task  even  though  analyses  of  customary 
performance  measures  failed  to  clearly 
discriminate  between  Instructional 

treatments.  Pilots  were  In  general  agreement 
that  the  freeze  added  to  the  difficulty  of  the 
task  and  suggested  that  this  difficulty  is.  In 
part,  traced  to  the  task  of 
"comlng-off -freeze,"  a  task  that  becomes  more 
difficult  the  closer  the  freeze  occurs  to 
touchdown.  It  thus  appears  that  the  effect  of 
the  freeze  is  more  than  that  of  simply  halting 
the  simulation.  Instead,  It  seems  to  add  a 
new  task  component  and  to  contribute,  as  well, 
to  the  overall  difficulty  of  the  primary  task 
being  learned. 

Lastly,  from  a  methodological  standpoint, 
the  present  study  Is  significant  In  that  It 
demonstrates  the  utility  of  the  probe  trial 
technique  as  an  alternative  to  the  more 
traditional  transfer  of  training  methodology 
for  the  preliminary  Investigation  of 
Instructional  treatment  effects.  In  the 
present  case,  the  probe  technique  proved  to  be 
sensitive  to  learning  effects  as  well  as  to 
subtle  differences  In  criterion  performance 
»rquwj  as  vt'e  ‘■esu’*  o*  *he  use  o"  the 
freeze. 

Recomnendatlons 

1.  Avoid  use  of  the  flight  simulator's  FREEZE 
feature  during  the  performance  of  a  continuous 
control  task  such  as  that  Involved  In  the 
approach  to  landing  task.  Other  tasks  to 
which  this  advice  might  also  apply  are  aerial 
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refueling  training  and  weapons  delivery 

training. 

2.  Consider  as  an  alternative  to  use  of  the 
FREEZE  the  use  of  the  simulator's  IN-FLIGHT 
CONDITION  STORE  feature  whereby  events 

captured  at  discrete  points  In  a  maneuver  can 
be  stored  and  later  recalled  for  student 
review.  Consider  also  use  of  the  simulator's 
RECORD/PLAYBACK  feature.  In  doing  so, 

however,  remember  that  In  some  Instances  the 

training  time  consulted  In  using  the  PLAYBACK 
may  be  better  used  to  provide  the  student 
additional  practice  time.  (7) 

1.  Response  chaining  as  a  training 
methodology  Is  not  ruled  out  by  the  findings 
of  the  present  study.  The  data  would  simply 
suggest  that  once  a  response  sequence  Is  begun 
it  Is  advisable  to  allow  the  sequence  to 
proceed  to  completion.  This  Is  true  whether 
chaining  Is  conducted  In  a  forward  direction 
wifi  termination  of  the  sequence  occurring  as 
the  result  of  an  error,  or  whether  chaining  Is 
conducted  In  a  "backward"  direction  (2)  where 
termination  occurs  with  performance  of  the 
f1nal  link  of  the  chain. 

4.  The  probe  methodology  is  strongly 

recommended  as  an  alternative  to  the 

traditional  transfer  of  training  paradigm, 

especially  for  exploratory  studies  where 
training  effectiveness  may  vary  not  only  as  a 
function  of  Instructional  approach  but  also  as 
a  function  of  amount  of  training. 
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Appendix 

Student  Pilot  Questionnaire 

INSTRUCTIONS:  Circle  the  number  on  the  scale 
that  best  describes  your  response  to  each  of 
the  following  Items. 

1.  Use  of  the  freeze  feature  may  be  used  to' 
significantly  decrease  the  overall  training 
time  required  to  learn  to  landing  task. 


1 

Strongly 

Disagree 


3 

Neutral 


Strongly 

Agree 


2.  Regaining  control  of  the  simulator 
following  a  freeze  slgnflcantly  added  to  the 
difficulty  of  the  flying  task  In  the  simulator 
(when  responding,  consider  each  of  the 
following  phases  of  the  maneuver  separately): 


(a)  "In  the  Middle"  (first  1/3) 
12  3  4 

Neutral 


Strongly 

Disagree 


Strongly 

Disagree 


(b)  "In  the  Groove"  (second  1/3) 
12  3  4 

Neutral 


Strongly 

Agree 


Strongly 

Agree 


(c)  "In  Close"  (5-10  seconds  from  the  ramp) 
1  2  3  4  5 


Strongly 

Disagree 


Neutral 


(d)  "At  the  Ramp" 

1  2  3 


Strongly 

Disagree 


Neutral 


Strongly 

Agree 


Strongly 

Agree 


4.  Improvements  In  performance  were  highly 
correlated  with  a  decrease  In  the  number  of 
freezes. 


1 

Strongly 

Disagree 


3 

Neutral 


Strongly 

Agree 


5.  Using  the  freeze  feature  to  explicitly 
identify  pilot  errors  during  the  "training" 
trials  made  It  easier  to  detect  errors  on 
"test"  trials  when  no  feedback  was  given  and 
when  no  freezes  were  In  effect. 


Strongly 

Disagree 


Neutral 


Strongly 

Agree 


6.  Compared  with  the  usual  practice  of  giving 
detailed  feedback  at  the  conclusion  of  a  task, 
providing  feedback  limedlately  following  an 
error  Is  more  effective. 


1 

Strongly 

Disagree 


3 

Neutral 


Strongly 

Agree 


7.  In  learning  the  task,  I  was  more  motivated 
by  trying  to  avoid  a  freeze  than  by  trying  the 
fly  the  task  correctly. 


1 

Strongly 

Disagree 


3 

Neutral 


Strongly 

Agree 


3.  The  occurrence  of  the  freeze  was 
"frustrating"  early  In  training. 


Strongly 

Disagree 


3 

Neutral 


Strongly 

Agree 


9.  A  helpful  feature  would  be  to  present  a 
"warning"  signal  (such  as  an  auditory  tone) 
prior  to  freezing  the  visual  system. 


1 

Strongly 

Disagree 


3 

Neutral 


Strongly 

Agree 


10.  Night  approaches  were  more  difficult  to 
Team  than  the  day  approaches. 


1 

Strongly 

Disagree 


3 

Neutral 


Strongly 

Agree 


11.  Errors  were  more  difficult  to  detect 
during  the  night  approaches  than  during  the 
day  approaches. 


1 

Strongly 

Disagree 


2 


3 

Neutral 


Strongly 

Agree 


12.  "Errors  serve  little  purpose,  since 
students  may  actually  learn  the  errors  that 
f.hev  commit..11 


1 

Strongly 

Disagree 


3 

Neutral 


Strongly 

Agree 


13.  "Errors  help  the  student  to  focus  on  the 
critical  elements  of  task  performance." 


Strongly 

Disagree 


3 

Neutral 


Strongly 

Agree 
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11.  "Instructional  methods  that  allow  errors 
to  freely  occur  are  inefficient,  since 
students  spend  valuable  time  practicing 
Incorrect  responses." 


1 

2 

3 

4 

5 

Strongly 

Disagree 

Neutral 

Strongly 

Agree 

IS.  "In  committing  errors,  students  learn  how 
to  recover  from  situations  which  at  some  later 
time  may  be  caused  not  by  task-specific  errors 
but  by  conditions  beyond  their  control  (for 
example,  by  adverse  weather,  visibility, 
turbulence,  etc.") 


1  2 

3  4 

5 

Strongly 

Disagree 

Neutral 

Strongly 

Agree 

16.  "Pointing  out  'errors' 

students,  whereas  pointing  out  what 
’S  dotng  'right'  Is  reinforcing." 

frustrates 
a  student 

1  2 

3  4 

5 

Strongly 

Disagree 

Neutral 

Strongly 

Agree 

17.  "Correct 

performance  results  from  a 

process  of  ellmlnlnatlng  errors  and  not  from 
a  process  of  shaping-desired  performance." 


1 

2  3  4 

5 

Strongly 

Disagree 

Neutral 

Strongly 

Agree 

18.  "A 

student's  recognition  of 

what  is 

considered  correct  Is  dependent  upon  his  being 
able  to  recognize  what  is  Incorrrect  (that  Is, 
an  error)." 


1 

2 

3 

4 

5 

St-ongly 

Disagree 

Neutral 

Strongly 

Agree 
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A  COMPARATIVE  ANALYSIS  OF  THREE  APPROACHES 
TO  THE  MAN-MACHINE  INTERFACE  PROBLEM 
AT  THE  FLIGHT  SIMULATOR  INSTRUCTOR/OPERATOR  STATION 


Joseph  L.  Dickman 
Sperry  Systems  Management  -  SECOR 
Fairfax,  Virginia 


ABSTRACT 

This  paper  analyzes  the  principles  of  instructor/operator  station  (IOS)  design  employed 
in  three  flight  simulators  recently  produced  by  the  same  company,  Sperry  Systems  Management  - 
SECOR.  The  analysis  brings  out  major  differences  among  the  three;  comments  on  the  reasons 
for  the  various  approaches  selected;  examines  the  experience  of  users  of  each  trainer,  to 
the  extent  that  data  is  available;  and  draws  conclusions  regarding  the  comparative  merits 
of  each  approach.  In  addition,  it  discusses  the  problem  of  objectively  evaluating  the 
efficiency  of  IOS  designs  in  general. 


INTRODUCTION 

Concurrent  with  other  trends  in  flight  simu¬ 
lator  development,  e.g.,  improving  the  fidelity 
of  aerodynamic  simulation,  increasing  the  detail 
and  realism  of  visual  scenes,  and  increasing 
the  variety  and  sophistication  of  instructional 
programs,  there  has  been  growing  interest  in 
improving  the  efficiency  of  the  instructor/ 
operator  station  (IOS).  There  is  new  awareness 
of  the  fact  that  a  knowledgeable  instructor, 
assisted  by  easy-to-interpret  displays  and 
easy-to-operate  controls,  is  central  to  the 
effectiveness  of  a  trainer. 

In  the  past  two  years  Sperry  Systems  Manage¬ 
ment  -  SECOk  has  produced  three  trainers — one 
is  still  to  be  delivered — whose  instructor- 
operator  stations,  for  various  reasons,  differ 
widely  in  design.  These  trainers  are  Device 
2F119,  the  EA-6B  Weapon  System  Trainer  (WST); 
Device  2F122,  the  A-6E  Night  Carrier  Landing 
Trainer  (NCLT);  and  Device  2F132,  the  F/A-18 
Operational  Flight  Trainer  (OFT). 

Each  of  these  trainers  has  a  remote  IOS, 
i.e.,  one  in  which  the  instructor's  console 
is  located  outside  the  cockpit  and  the  instructor 
monitors  student  performance  with  his  own 
displays,  indicators,  and  instruments  rather  than 
by  direct,  over-the-shoulder  observation.  In 
other  respects,  hdWever,  each  instructor  station 
is  significantly  unique,  and  each  presents  a 
different  approach  to  the  problems  of  man-machine 
interface  that  occur  at  the  IOS. 


EA-6B  WST 

The  EA-6B  WST  is  located  at  NAS  Whidbey 
Island,  Washington,  and  achieved  final  Government 
acceptance  in  December,  1979.  Its  primary  mission 
is  to  train  the  members  of  the  four-man  crew  in 
all  aspects  of  a  combat  mission,  including  pre¬ 
flight  preparations,  engine  start,  carrier  launch, 
navigation  to  a  target  area,  penetration  of 
enemy  defenses,  utilization  of  electronic  jamming 
equipment,  defense  against  hostile  counter-action, 
return  to  the  carrier,  and  landing.  Most  of  the 
utilization  of  the  trainer,  however,  is  for 
training  pilots  in  emergency  procedures,  instru¬ 
ment  flight,  carrier  landings,  and  such  operations 


that  do  not  include  a  full  crew.  In  addition, 
the  trainer  is  used  for  training  ECM  operators 
(ECMOs )  alone,  and  supplements  a  part-task 
trainer,  the  15E22A,  which  is  available  at  Whidbey. 


As  is  fitting  for  a  trainer  with  a  complex 
mission,  the  instructor  station  is  quite  large. 

With  a  wide  wrap-around  configuration  (see 
Figure  1),  it  is  designed  to  accommodate  three 
instructors — one  (designated  the  Flight  Instructor) 
for  monitoring  the  pilot,  one  (the  ECMO-1  Instruc¬ 
tor)  for  the  ECMO-1,  and  the  third  (the  Tactics 
Instructor)  for  the  ECMO-2  and  ECMO-3.  In  the 
aircraft,  the  ECMO-1,  who  sits  ir.  the  right  front 
seat,  operates  the  navigation  radar  and  the 
ALQ-92  communications  jammer;  the  ECMO-2  and 
ECMO-3,  who  sit  in  the  aft  cockpit,  operate  the 
ALQ-99  tactical  jamming  system. 

The  instructor  station  is  equipped  with 
five  alphanumeric-graphic  CRTs  and  a  visual 
monitor.  Two  of  these  CRTs  and  the  visual 
monitor  are  intended  for  the  Flight  Instructor, 
and  he  can  also  use  a  third  CRT,  which  is  in 
front  of  the  ECMO-1  Instructor  position. 

Each  instructor  has  a  row  of  control  panels 
containing  an  assortment  of  push-button,  thumb¬ 
wheel  and  other  switches.  At  the  Flight 
Instructor  position  there  are  the  Visual  System 
Control  Panel,  with  controls  for  initiating  and 
varying  certain  visual  effects  (but  not  all); 
the  Mission  Control  Panel,  by  which  the  instructor 
selects  and  enters  initial  conditions,  demonstra¬ 
tions,  and  "programmed  missions";  the  Trainer 
Control  Panel,  with  controls  for  operating  the 
motion  system,  varying  sound,  ground  roll  and 
rough  air,  initiating  starting  air  and  external 
power,  removing  chocks,  operating  trainer  freeze 
and  crash  override,  and  performing  other  similar 
functions;  and  the  Aircraft/Conmunication  Control 
Panel,  containing  controls  for  communicating  with 
the  students  and  other  instructors.  Miscellaneous 
controls  for  other  functions  not  mentioned  above 
are  also  included  on  these  panels.  There  are, 
in  total,  47  operative  controls  on  the  four  panels. 

In  addition  to  the  various  control  panels, 
each  instructor  has  a  function  keyboard  consisting 
of  32  illuminated  but  untitled  keys.  The  titles 
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of  the  keys,  defining  their  functions,  are  con¬ 
tained  on  an  overlay  which  the  instructor  inserts 
at  the  beginning  of  the  mission.  There  are  six 
different  overlays  available  for  the  Flight 
Instructor,  one  for  each  of  the  principal  modes 
of  operation:  Operate,  Plan  (Disc  File  Genera¬ 
tion),  Plan  (Demonstration),  Display  Printout, 
Critique,  Test,  and  DRED  (DRED  is  an  acronym 
for  Daily  Readiness  Testing).  The  Tactics 
Instructor  has  a  unique  set  of  overlays;  the 
ECMO-1  Instructor  uses  overlays  identical  to 
those  for  the  Flight  Instructor. 

When  the  Operate  overlay  is  installed  at 
the  Flight  Instructor's  position,  the  keys  are 
used  for  calling  up  displays  and  for  performing 
certain  control  functions,  such  as  storing  CRT 
displays  for  later  print-out,  enabling  map 
slew  and  aircraft  slew  (operated  in  conjunction 
with  a  "joystick"  control),  deleting  the  display 
of  radio  facilities  not  tuned  in  by  the  pilot, 
and  overriding  imminent  malfunctions.  Half  of 
the  keys  are  used  directly  for  calling  up 
displays;  the  remaining  half  are  for  control 
functions . 

The  use  of  overlays  at  the  IOS  is  not 
standard  in  Sperry  SECOR's  design  philosophy. 

The  function  keyboard  installed  in  the  EA-6B  WST, 
and  its  attendant  collection  of  overlays,  was 
chosen  in  order  to  maintain  consnonality  with 
the  sub-contractor  (the  AAI  Corporation)  who 
was  providing  the  Tactics  part  of  the  trainer, 
derived  from  their  part-task  ECM  trainer,  the 
15E22A.  In  fact,  Sperry  SECOR  procured  the 
keyboard  from  AAI . 

Because  the  function  keyboard  has  a  limited 
number  of  keys,  it  cannot  accomnodate  all 
required  control  functions.  Some  are  assigned 
to  the  Trainer  Control  Panel.  Those  retained 
on  the  function  keyboard  are  ones  with  the 
closest  relationship  to  CRT  displays,  since 
management  of  displays  is  the  principal  function 
of  the  function  keyboard. 

It  would  not  have  been  practical  to  use  an 
additional  overlay  for  these  "overflow"  functions 
because  of  the  undesirability  of  requiring  the 
instructor  to  change  overlays  in  mid-flight, 
so  to  speak.  Changing  overlays  requires  going 
off-line  and  interrupts  the  mission.  On  the 
other  hand,  the  use  of  separate  overlays  for 
Plan,  Display  Printout,  Test  and  the  other  modes 
is  not  an  inconvenience  because  a  natural 
interruption  of  the  mission  continuity  occurs 
preceding  those  activities. 

To  complete  the  array  of  controls  at  the 
IOS,  an  alphanumeric  keyboard  and  a  light  pen 
are  provided  for  each  instructor.  Also,  each 
instructor  has  a  CRT  Select  Switch  for  desig¬ 
nating  the  CRT  where  a  display  is  to  appear. 


Displays 


The  EA-6B  WST  has  an  exceptionally  extensive 
library  of  displays.  There  are  ten  displays 
for  initial  conditions  sets  (one  for  each  set 
available),  three  displays  depicting  the  instru¬ 
ment  readings  and  control  or  switch  positions 
in  the  cockpit,  four  types  of  map  or  map-like 
displays  with  four  possible  scales,  13  displays 


listing  all  of  the  programmable  malfunctions 
(the  trainer  can  simulate  over  700  malfunctions, 
including  220  open  circuit  breakers),  98  displays 
of  normal  and  emergency  procedures,  a  display  to 
initiate  "parameter  recording",  a  series  of 
map  displays  for  monitoring  "computer-evaluated 
missions"*,  two  displays  listing  input  codes,  and 
several  other  specialized  displays  for  programming 
and  maintenance  purposes. 

The  input  code  displays  represent  one  of 
the  most  controversial  characteristics  of  the 
trainer — the  use  of  non-display  dependent  formats 
for  data  entry.  All  on-line  parameter  changes, 
such  as  increasing  or  decreasing  the  fuel  load, 
adding  or  removing  external  stores ,  varying  the 
wind  direction  and  velocity,  and  changing  the 
ceiling  and  visibility  in  the  visual  scene,  are 
accomplished  by  entering  an  input  code  followed 
by  a  numerical  value.  To  enter  a  surface  wind 
velocity  of  20  knots,  for  example,  the  instructor 
types  WB  20  with  the  alphanumeric  keyboard  and 
depresses  the  RETURN  key.  Any  entry  can  be 
made  regardless  of  whac  displays  are  on  the  CRTs, 
as  long  as  the  instructor  knows  the  proper  input 
code . 

A  total  of  112  input  codes  are  listed  on 
the  two  input  code  displays  (one  of  the  displays 
is  shown  in  Figure  2).  They  are  organized  into 
six  categories — Flight  Parameters,  Aircraft 
Configuration,  Environment,  Nav/Comm,  Visual,  and 
Others.  Most  of  the  codes  are  composed  of  two 
letters;  however,  there  are  eight  one-letter 
codes,  one  three-letter  code,  and  nine  four- 
letter  codes.  The  codes  are  designed  to  facilitate 
memorizing  as  much  as  possible;  typical  codes 
are  AA  for  angle-of-attack,  RC  for  rate  of  climb, 

LA  for  latitude,  D  for  display,  and  C  for  carrier. 
Most  visual  codes  start  with  V;  VC  is  for  ceiling, 
and  VI  for  visibility.  A  number  of  visual  codes 
related  to  intensity  of  lighting  start  with  I; 
examples  are  IT  for  intensity  of  taxiway  lights, 
and  IF  for  intensity  of  flood  lights.  Inevitably 
there  are  instances  where  the  best  code  for  a 
parameter  has  already  been  assigned  to  another; 
the  second  choice  is  not  always  as  logical.  For 
example,  IP  had  to  be  used  for  intensity  of 
droplights  since  ID  was  already  assigned  to 
intensity  of  deck  lights. 

Most  of  the,  one-letter  input  codes  are  used 
in  combination  with  the  two-letter  codes.  For 
example,  LA  and  LO,  used  to  change  the  latitude 
and  longitude  of  the  aircraft  ("ownship"),  apply 
to  the  carrier  when  prefixed  with  C;  and  when 
used  with  G  tney  will  relocate  the  center  of  the 
gaming  area.  The  letter  I,  when  used  as  a  prefix 
to  a  two-letter  code,  applies  to  the  parameters  in 


*Parameter  Recording  produces  a  continuous 
strip  chart  depicting  a  student’s  ability  to 
maintaxn  one  or  more  flight  parameters  within 
specified  limits — an  airspeed  of  140  knots  +  or  - 
10,  for  example;  a  Computer-Evaluated  Mission, 
also  called  a  Programnied  Mission,  automatically 
measures  student  performance  over  a  preplanned 
flight  profile  containing  a  number  of  legs,  or 
segments,  designed  to  facilitate  operation  of 
the  computer  program  without  instructor  inter¬ 
vention. 
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INPUT  CODES 


FLIGHT  PARAMETERS  ENVIRONMENT 


ANGLE  OF  ATT\CK  AA 
ANGLE  OF  BANK  AB 
ACCELERATION  AC 
ALTITUDE  AL 
INDICATED  AIRSPEED  AS 
HEADING  HD 
LATITUDE  LA 
LONGITUDE  LO 
MACH  NUMBER  MN 
PITCH  ANGLE  PA 
PITCH  RATE  PR 
RATE  OF  CLIMB  RC 
RATE  OF  DESCENT  RD 
ROLL  RATE  RO 
RPM  LEFT  RL 
RPM  BOTH  RP 
RPM  RIGHT  RR 
RATE  OF  TURN  RT 
TRUE  AIRSPEED  TA 
YAW  RATE  YR 

AIRCRAFT  CONFIGURATION 

STATION,  CENTERLINE  CL 
EMERGENCY  LANDING  GEAR  EG 
AIR  REFUELING  ENABLED  FE 
FLAPS  FL 
FUEL  LOADING  FU 
STATION,  LEFT  INBOARD  IL 
STATION,  RIGHT  INBOARD  IR 
LANDING  GL-iR  POSITION  LG 
STATION,  I, EFT  OUTBOARD  OL 
STATION,  RIGHT  OUTBOARD  OR 
SPEEDBRAKE  SB 


BAROMETRIC  PRESSURE 

WIND  DIRECTION  AT  20000  FT 

ICING 

MANUAL  MAGNETIC  VARIATION 
MAGNETIC  VARIATION 
RAINSHOWER  DENSITY 
RUNWAY  SURFACE 
SEA  STATE 

TEMPERATURE,  FREE  AIR 
WIND  DIRECTION  AT  SURFACE 
WIND  VELOCITY  AT  20000  FT 
RADAR  WEATHER  SET 
WIND  VELOCITY  AT  SURFACE 

NAV/COMM 

ACLS  ELEVATION 
ADF  BEARING 
ACLS  CENTERLINE 
ACLS  AZIMUTH 
ARTCC  FREQ 
APC  FREQ 
NDB  ON/OFF 
GCA/CCA  AZIMUTH 
GCA/CCA  CENTERLINE 
GCA/CCA  ELEVATION 
RADIO  FACILITY  SET 
RUNWAY  IN  USE 
TACAN  BEARING 
TACAN  DME 
TERMINAL  AREA 
TACAN  ON/OFF 


Fiqure  2.  Input  Codes  Display 
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an  initial  condition*  aet.  For  example,  by 
entering  ILA  and  1L0,  followed  by  the  appropriate 
numerical  values,  the  instructor  can  change  the 
latitude  and  longitude  of  the  ownship  as  indicated 
on  any  initial  conditions  set  being  displayed. 

The  letter  B  is  used  to  activate  malfunc¬ 
tions.  The  instructor  enters  B  followed  by  the 
number  of  the  malfunction,  which  he  can  obtain 
from  the  displays  listing  all  available  malfunc¬ 
tions,  unless  he  remembers  the  number  or  has 
previously  determined  it  in  his  pre-mission 
planning.  To  resiove  the  malfunction  he  enters  R, 
followed  again  by  the  number. 

Until  development  of  the  EA-6B  WST,  Sperry 
SECOR  had  never  used  input  codes  and  non-display 
dependent  formats,  except  for  specific  purposes 
such  as  entering  malfunctions.  The  rationale 
for  adopting  input  codes  for  the  EA-6B  WST  was, 
again,  to  be  consistent  with  AAI ,  whose  Device 
15E22A  uses  this  approach  (their  term  for  an 
input  code  is  "cotraand  word"). 

The  light  pen  is  available  as  an  alternative 
input  method  for  many  IOS  functions.  For 
example,  it  can  be  used  to  enter  malfunctions 
by  illuminating  the  number  of  the  desired  mal¬ 
function  on  the  Malfunction  Display,  and  to 
remove  the  malfunction  by  illuminating  the  title. 
It  is  the  only  method  for  activating  simulated 
antiaircraft  fire  and  surface-to-air  missiles 
via  the  Hostile  Environment  Display.  It  is 
also  the  most  flexible  method  for  removing  the 
dotted  aircraft  track  (to  reduce  clutter). 

User  Experience 

The  principal  user  of  the  EA-6B  WST  is 
VAQ  129,  a  squadron  at  NAS  Whidbey  Island  whose 
primary  mission  is  the  training  of  replacement 
pilots  and  ECMOs.  Currently  (June  1981) 
there  are  12  instructors  (four  pilots  and  eight 
ECMOs)  trained  in  the  operation  of  the  trainer. 

In  addition  there  are  eight  other  personnel  able 
to  function  as  device  operators. 

When  a  full-crew  mission  is  flown  in  the 
trainer,  the  instructor  station  is  usually 
manned  by  an  instructor  at  each  of  the  Flight 
and  Tactics  Instructor  positions,  a  device 
operator  at  the  ECMO-1  Instructor  position,  and  a 
supervisory  instructor  seated  or  standing  behind 
the  other  three  persons.  The  use  of  a  device 
operator  has  evolved  since  the  trainer  was 
delivered.  The  original  Specification  for  the 
trainer  did  not  call  for  that  position. 

Because  the  ECMO-1  Instructor  position 
does  not  have  a  full  set  of  controls  (there  is 
no  Mission  Control  Panel  and  the  Trainer  Control 
Panel  is  considerably  abbreviated),  the  device 
operator  seated  there  has  limited  utility. 

However,  he  has  a  function  keyboard  and  an 
alphanumeric  keyboard,  and  he  can  call  up  displays 
on  his  CRT  and  take  actions  involving  use  of 
the  input  codes,  including  entering  and  removing 
malfunctions  (at  the  Flight  Instructor's 
direction).  Since  he  has  a  CRT  Select  Switch, 
the  device  operator  can  generate  displays  on  the 
Flight  Instructor 's  two  CRTs,  but  in  practice 
the  Flight  Instructor  handles  his  own  displays. 

The  device  operator  cannot  assist  both  the 


Flight  ind  Tact  ts  Instructors  because  he  would 
have  to  use  two  Operate  overlays  on  his  function 
keyboard — an  impractical  procedure,  as  explained 
previously. 

In  general,  the  users  of  the  EA-6B  WST 
are  satisfied  with  the  design  philosophy  of  the 
IOS.  They  have  no  significant  adverse  opinions 
regarding  the  function  keyboard  and  overlays, 
the  non-display  dependent  formats  and  input  codes, 
and  the  alphanumeric  keyboard.  Since  they 
expect  the  device  operator  to  be  the  principal 
user  of  the  alphanumeric  keyboard,  they  are  not 
concerned  about  the  reputed  antipathy  of 
instructors,  particularly  pilots,  toward  typing. 
They  agree  with  the  need  for  consistency  between 
the  Flight  and  Tactics  sides  of  the  IOS,  and 
being  familiar  with  AAl's  designs  in  Device  15E22A, 
they  accept  those  approaches  as  being  the  appro¬ 
priate  ones  to  use. 

They  rarely  use  the  light  pen  as  an  input 
device,  except  when  it  is  the  only  method  avail¬ 
able.  A  major  reason  for  the  light  pen's 
unpopularity  is  the  fact  that  it  is  relatively 
slow  and  difficult  to  aim  precisely  at  a  desired 
point  on  the  display. 

Also,  the  users  are  dissatisfied  with  the 
Pilot's  Instrument  Monitor  Display.  This 
display  depicts  the  various  instruments  in  their 
relative  positions  on  the  aircraft  instrument 
panel,  but  the  readouts  are  digital  with  a  one- 
hertz  update  rate.  The  users  are  considering 
adding  a  number  of  repeater  instruments  to  the 
instructor  station,  or  substituting  for  the 
digital  display  a  pseudo-instrument  display  with 
analog  readouts. 

The  Flight  Instructors  and  Tactics  Instructors 
who  are  experienced  in  use  of  the  trainer  do  not 
have  difficulty  in  coping  with  the  complexity  of 
the  IOS.  However,  they  are  limited  in  number, 
considering  the  intense  training  program  at 
Whidbey,  and  are  being  subjected  to  continual 
turnover.  The  trainer  is  too  complex  to  be 
operated  by  unqualified  personnel.  The  device 
operators  are  useful,  especially  during  trainer 
turn-on  and  problem  set-up;  however,  instructors 
must  be  thoroughly  familiar  with  the  trainer's 
capabilities  in  order  to  give  effective  direction 
to  the  device  operator. 


A-6E  NCLT 

Two  A-6E  NCLTs  have  been  delivered,  one  to 
NAS  Whidbey  Island,  Washington,  and  one  to  NAS 
Oceana,  Virginia.  The  acceptance  dates  were 
December  1980  and  March  1981,  respectively.  The 
primary  mission  of  the  NCLT  is  to  train  pilots  in 
carrier  landings.  It  can  also  provide  training  in 
launch  procedures  and  in  a  limited  number  of 
emergency  procedures. 

IOS  Configuration 

The  instructor  station  is  designed  to 
accommodate  an  instructor  and  a  device  operator 
(see  Figure  3),  while  complying  with  the  stipula¬ 
tion  in  the  Specification  that  the  IOS  must  be 
able  to  be  operated  by  one  person.  There  are 
two  CRTs,  one  for  each  position,  and  a  visual 
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INSTRUCTOR  STATION 


Figure  3.  A-6E  NCLT  Instructor/Operator  Station 
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monitor  for  the  instructor.  An  alphanumeric  key¬ 
board  is  located  in  front  of  the  device  operator, 
and  an  LSO  (Landing  Signal  Officer)  hand  control 
is  provided  for  the  instructor. 

There  are  two  control  panels.  One,  called 
the  Left-Hand  Control  Panel  and  located  below 
the  Visual  Monitor,  contains  an  indicator  to 
warn  of  a  malfunction  in  the  simulated  oxygen 
system  in  the  cockpit,  a  "beeper"  to  announce 
an  over-temperature  in  the  computer  area  or 
at  the  student  station,  three  push-button 
switches  related  to  the  visual  system,  and 
three  rotary  controls  for  varying  the  volume 
in  two  of  the  IOS  headsets  and  in  the  intercom 
speaker.  The  other,  the  Function  Panel, 
contains  five  switches  and  indicators  related 
to  the  motion  system,  three  conmunications 
switches,  and  eight  function  switches.  The 
function  switches  include  those  for  launch, 
freeze,  reset,  playback,  etc.  The  Function 
Panel  is  located  below  the  instructor's  CRT, 
thus  the  device  operator  has  only  the  alpha¬ 
numeric  keyboard. 

Displays 

The  NCLT's  repertoire  of  displays  is  rela¬ 
tively  limited,  containing  a  total  of  23 
displays.  The  first  display  is  an  index  6f 
all  subsequent  ones  (the  displays  are  called 
pages  and  the  index  page  is  identified  as  PO). 

Tb~  next  display,  PI,  is  the  Program  Mode 
Control  Page,  and  it  is  used  for  entering  the 
principal  modes  of  operation:  Mission,  DRED, 
Display  Printout,  and  Demo.  It  also  enables  a 
number  of  specific  functions,  such  as  previewing 
displays  before  printing,  clearing  stored  dis¬ 
plays  from  the  memory,  and  recording  for  a 
demonstration. 

P2  is  the  Initial  Conditions  Page;  it  lists 
the  principal  parameters  for  five  sets  of 
initial  conditions.  With  the  alphanumeric 
keyboard,  the  instructor  can  change  the  values 
of  the  parameters  before  any  set  is  entered; 
he  enters  the  desired  set  by  depressing  the 
proper  switch  on  the  Function  Panel. 

P3  is  the  Monitor  Control  Page;  it  enables 
the  instructor  to  monitor  the  position  of 
certain  controls  in  the  cockpit,  such  as  the 
throttles,  the  air  start  buttons,  the  RAT  handle, 
and  the  arresting  hook.  It  also  allows  the 
instructor,  on-line,  to  modify  parameters,  such 
as  fuel  quantity,  external  tank  configuration, 
wind  speed  and  direction,  and  carrier  speed  and 
heading. 

Other  displays  (pages)  include  Visual  System 
Control,  Performance  Evaluation,  Malfunctions, 
Instruments,  Area  Map,  CCA,  Parameter  Recording, 
and  13  displays  for  monitoring  selected  emergency 
procedures.  Forty-eight  malfunctions  generally 
related  to  carrier  launch  and  recovery  operations 
are  programmable. 

On-line  parameter  changes  are  accomplished 
with  display-dependent  formats,  which  is  a 
major  difference  between  the  A-6E  NCLT  and  the 
EA-6B  WST.  To  change  the  fuel  quantity,  for 
example,  the  instructor  calls  up  the  Monitor 
Control  Page,  if  it  is  not  already  displayed  on 


one  of  the  two  alphanumeric/graphic  CRTs;  insures 
that  the  CRT  Select  Switch  is  indicating  the 
proper  CRT;  types  the  line  number  for  FUSELAGE 
FUEL  QTY  (in  this  case  the  instructor  must 
decide  whether  to  increase  the  fuel  in  the 
fuselage  tank,  wing  tanks,  or  external  tanks); 
types  a  comma  as  a  separator;  and  then  types 
the  desired  value  of  the  parameter.  Finally  he 
depresses  the  RETURN  key  on  the  alphanumeric 
keyboard  to  complete  the  entry.  To  fill  the 
fuselage  tank  to  9000  lbs,  the  complete  entry  is 
"1,9000",  plus  RETURN. 

There  are  five  pages  containing  numbered 
lines  enabling  parameter-changing’  the  Program 
Mode  Control  Page,  the  Initial  Conditions  Page, 
the  Monitor/Control  Page,  the  Visual  System 
Control  Page,  and  the  Performance  Evaluation 
Page.  Some  of  these  pages  also  contain  unnumbered 
lines  which  are  for  monitoring  purposes  only, 
rather  than  control  (i.e.  parameter-changing). 

On  the  Monitor/Control  Page  approximately  half  of 
the  lines  are  for  monitoring  only  (see  Figure  4). 

The  Malfunctions  Page  lists  the  48  available 
malfunctions  and  assigns  each  a  number.  However, 
the  method  of  activating  a  malfunction  is  to 
type  the  letter  A  plus  the  appropriate  number 
(followed  by  the  RETURN  key).  To  remove  a  mal¬ 
function,  the  letter  R  is  used  similarly.  These 
actions  can  be  taken  at  any  time,  regardless  of 
what  pages  are  displayed,  as  long  as  the  user 
knows  the  number  of  the  desired  malfunction.  This 
approach  resembles  the  input  codes  of  the  EA-6B 
WST;  the  reason  for  such  a  deviation  from  the 
other,  display-dependent  formats  of  the  NCLT  is 
to  give  the  instructor,  or  the  device  operator, 
more  flexibility. 

The  Index  Page  is  used  in  a  similar  way. 

If  the  instructor  or  device  operator  knows  the 
number  of  the  display  that  he  wants  to  view,  he 
can  obtain  it  by  typing  P  and  the  number  without 
having  the  Index  Page  displayed. 

Sperry  SECOR's  rationale  in  selecting  the 
IOS  design  used  in  the  NCLT  was  that  it  was 
simple  and  proven  (a  similar  approach  had  been 
used  by  the  same  company  on  previous  trainers). 
Also,  there  were  no  external  factors  influencing 
the  choice,  such  as  maintaining  commonality 
with  another  contractor. 

User  Experience 

The  A-6E  NCLT  is  used  by  a  number  of  attack 
squadrons  at  Whidbey  and  Oceana.  At  the  two 
stations  there  is  currently  a  total  of  approxi¬ 
mately  15  instructors  and  15  device  operators 
trained  to  operate  the  trainer. 

Normally  both  an  instructor  and  a  device 
operator  are  used  during  trainer  exercises  (it 
should  be  noted  that  the  trainer  has  been 
operational  at  each  location  a  limited  amount  of 
time).  The  instructor  is  able  to  devote  most  of 
his  attention  to  monitoring  student  performance 
while  the  device  operator  is  concerned  primarily 
with  using  the  alphanumeric  keyboard.  The 
instructor  also  uses  the  switches  on  the  Left  Hand 
Control  Panel  and  the  Function  Panel,  but  this 
responsibility  is  not  too  demanding.  The  motion 
system  switches,  which  are  on  the  function  panel, 
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are  sometimes  operated  by  the  device  operator, 
but  he  can  reach  them  easily  with  his  left  hand. 
While  individual  preferences  vary,  usually  the 
instructor  keeps  the  Instruments  Page  on  the 
left-hand  CRT,  leaving  the  other  CRT  for  all 
other  displays. 

At  each  location,  copies  of  the  Index  Page 
and  the  Malfunctions  Page  have  been  taped  to  the 
top  of  the  work  surface.  This  practice  may 
offend  purists  but  it  is  an  effective  way  of 
avoiding  tedious  display-manipulation.  It  can 
be  expected,  furthermore,  that  device  operators 
will  quickly  memorize  most  of  the  page  numbers. 

In  general,  the  users  of  the  NCLT  are 
satisfied  with  their  IOS.  They  have  no  reason 
to  desire  non-display  dependent  formats,  par¬ 
ticularly  since  the  principal  disadvantage  of 
display-dependent  formats  has  been  solved  by  the 
expedient  cited  above,  as  well  as  by  having,  in 
effect,  input  codes  for  entering  and  removing 
malfunctions  and  calling  up  display  pages. 

The  users  at  Whidbey  have  a  criticism  of 
the  parameter  recording  procedure  which  they 
intend  to  correct  with  an  ECP.  Parameter 
recording  is  a  two-minute  program  for  recording 
and  graphically  displaying  student  performance 
during  a  carrier  approach. 

The  procedure  for  starting  the  recording 
requires  that  the  CRT  page  be  displayed  while 
the  device  operator  enters  the  line  number  for 
START  PECORDING  and  a  "1"  for  activation.  This 
process  occupies  the  CRT,  although  briefly,  during 
a  critical  part  of  the  exercise;  the  instructor 
normally  wants  other  displays  (the  Instruments 
Page  and  the  GCA/CCA  Page,  for  example)  on  the 
two  CRTs  at  this  time.  What  the  users  want  is 
simply  a  function  switch  to  start  the  recording. 

F/A-18  OFT 

The  F/A-18  OFT,  Device  2F132,  is  scheduled 
to  be  operational  at  NAS  Lemoore,  California, 
in  March  1982.  An  additional  trainer  will  be 
delivered  to  NAS  Cecil  Field,  Florida,  approxi¬ 
mately  two  years  later.  Typical  of  OFTs ,  the 
primary  mission  of  Device  2F132  is  to  train  pilots 
in  ail  aspects  of  instrument  and  visual  flight 
except  tactics  and  weapon  delivery  (the  trainer 
has  a  limited  air-to-ground  weapon  delivery 
capability,  however). 

IOS  Configuration 

The  instructor  station  is  a  radical  departure 
from  conventional  approaches.  As  described  in  the 
Trainer  Configuration  Report,  the  concept  of  the 
design  of  the  IOS  is  to  provide  the  instructor 
"the  means  to  conduct  and  monitor  the  training 
mission  with  visual  cues  that  are  familiar  to  him 
in  the  F/A-18  cockpit".  With  this  objective 
specifically  in  mind,  the  F/A-18  OFT's  IOS  is 
configured  so  as  to  resemble  the  F/A-18  cockpit 
as  much  as  practicable.  The  visual  monitor  is 
placed  in  the  center  and  an  alphanutneric/graphic 
CRT  is  located  on  each  side  (see  Figure  5).  A 
repeater  of  the  cockpit  Head-Up-Display  (HUD)  is 
superimposed  on  the  visual  monitor;  and  repeaters 
of  the  Master  Monitor  Display  (MMD),  the  Multi- 


Function  Display  (MFD),  and  the  Horizontal 
Situation  Display  (HSD) ,  which  are  six-inch  CRTs, 
called  Digital  Display  Indicators  (DDIs),  located 
on  the  front  instrument  panel  in  the  cockpit, 
are  located  at  the  IOS  below  the  visual  monitor. 
The  HUP  is  the  primary  flight  instrument  in  the 
aircraft,  and  the  MMD,  MFD,  and  HSD  are  the 
primary  system  and  navigation  displays.  Thus  the 
presentation  of  these  displays  at  the  IOS  is 
optimized  for  the  instructor,  who  will  normally  be 
devoting  much  of  hiB  attention  to  the  visual 
scene . 

The  instructor  will  be  seated  in  front  of 
the  visual  monitor,  and  the  device  operator, 
when  utilized,  will  be  seated  to  the  instructor's 
right.  When  the  instructor  is  alone  at  the  IOS, 
both  CRTs,  which  are  rotatable,  will  normally 
be  turned  inward;  when  the  device  operator  is 
also  present,  the  right-hand  CRT  will  be  turned 
ao  that  It  is  facing  to  the  front  and  can  be  more 
easily  viewed  by  him. 

Nine  function  keys  are  grouped  around  the 
visual  monitor.  They  include  freeze,  reset, 
malfunction  override,  crash  override,  normal 
arrest ,  barrier  arrest,  and  other  such  functions 
requiring  rapid  access  by  the  instructor.  The 
remaining  controls,  which  are  quite  limited  in 
number,  are  arranged  on  panels  below  the  two 
alphanumeric/graphic  CRTs.  The  instructor  has 
three  function  keys  for  communications ;  the 
device  operator  has  two  for  varying  sound  volume 
and  lamp  intensity.  Each  person  has  a  12-key 
keypad;  a  function  key  for  calling  up  the  Menu 
Display,  which  is  the  equivalent  of  an  Index 
Display;  and  a  function  key  for  saving  any 
display  for  later  printing.  The  instructor  has 
a  "joystick"  for  control  of  map  slew  and  aircraft 
slew. 

Each  alphanumeric/Graphic  CRT  has  a  light 
pen  and  the  visual  monitor  has  an  LSO  hand 
control.  An  alphanumeric  keyboard  is  available 
at  the  device  operator's  position  for  non-training 
functions,  such  as  activating  plan  Mode  and  DRED 
and  "debugging"  software. 

Displays 

Basically,  the  F/A-18  OFT  uses  display- 
dependent  formats  activated  by  the  light  pen. 

The  displays  are  designed  to  progress  from 
general  to  specific,  so  that  the  instructor  or 
device  operator,  in  effect,  merely  follows 
"go  to"  instructions  on  each  page.  A  back-up 
method,  enabling  activation  via  the  keypad  instead 
of  the  light  pen,  is  available;  it  usee  numbered 
lines  and  entry  formats  identical  to  those  on 
the  A-6E  NCLT. 

The  initial  display  is  the  Menu  Display 
(the  title  comes  from  a  similar  display  in  the 
F/A-18  aircraft);  it  is  displayed  automatically 
on  both  CRTs  when  the  computer  is  initialized, 
and  can  be  called  up  at  any  time  by  the  Menu 
Display  Switch,  which  is  located  on  each  of  the 
panels  below  the  two  CRTs. 

The  Menu  Display  (see  Figure  6)  enables  the 
instructor  to  call  up  a  number  of  index  displays: 
Mode  Selection,  Map  Displays,  Malfunctions, 
Procedures,  Cockpit  Monitors,  Environment/Aircraft 
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MENU 


Figure  6.  Menu  Display 
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Data,  and  Programmed  Mission.  The  method  of 
operation  is  to  "illuminate"  with  the  light  pen  a 
small  square  (called  the  "strike  area")  to  the 
left  of  each  title.  If  the  light  pen  is  malfunc¬ 
tioning,  the  back-up  method  is  to  enter  the  line 
number  (1  for  Mode  Select,  2  for  Map  Displays, 
etc.)  with  the  keypad  and  depress  the  ENT  key  on 
the  keypad.  At  the  bottom  of  the  page,  and  on 
most  other  displays,  there  is  a  line  entitled 
ASSIGN  OTHER  CRT.  This  enables  the  instructor 
to  assign  the  display  to  the  device  operator's 
CRT,  and  vice  versa. 

The  organization  of  displays  is  intended  to 
carry  on  the  analogy  with  the  F/A-18  aircraft. 

In  the  aircraft,  the  MMD,  MFD,  and  HSD  have  a 
number  of  push-button  switches  around  their 
peripheries.  The  function  of  each  switch  varies 
according  to  the  display  in  use,  and  the  title 
appears  on  the  CRT  beside  the  switch.  This 
concept  is  repeated  on  the  trainer  displays;  the 
strike  ureas  are  intended  to  represent  the 
switches  and  the  light  pen  the  pilot's  finger. 

From  the  Menu  Display  the  instructor  goes 
to  the  Mode  Selection  Display  where  he  chooses  one 
of  seven  possible  modes:  Freef light,  Checkride, 
Automission,  Dynamic  Replay,  Minute  Replay,  Demo, 
and  Print.  Selecting  any  of  these  modes  leads 
to  its  own  hierarchy  of  displays.  For  example, 
if  the  instructor  selects  the  Freef light  Mode, 
the  Freef light  Mode  Initial  Conditions  Index  is 
displayed.  This  page  lists  25  sets  of  initial 
conditions  and  enables  the  instructor  either  to 
enter  immediately  the  set  desired  or  to  call  up 
a  descriptive  display  of  the  composition  of  any 
set,  i.e.,  a  list  of  the  aircraft  and  environmen¬ 
tal  parameters  and  the  navigational  waypoints 
programmed  into  the  on-board  computer.  The 
instructor  can  review  these  elements  but  he  is 
not  able  to  make  on-line  changes  to  an  initial 
conditions  set  before  entering  it. 

At  the  bottom  of  the  Freeflight  Initial 
Conditions  Index,  in  addition  to  the  ASSIGN 
OTHER  CRT  line,  are  two  other  lines  entitled 
COCKPIT  MONITOR  and  MAP  DISPLAYS.  These  lead 
the  instructor  to  the  indicated  displays,  which 
are  additional  indexes,  from  which  he  can  select 
appropriate  sub-displays. 

All  of  the  displays  have  similar  exit  lines, 
each  set  designed  to  provide  the  most  logical 
options  based  on  the  purpose  of  the  display  being 
slown.  If  the  instructor  reaches  a  dead  end, 
he  can  always  select  the  Menu  Display  Switch  on 
the  panel  below  his  CRT  and  go  Back  to  the 
beginning  of  the  display  structure. 

If  the  instructor  selects  the  Cockpit 
Monitor  Display,  the  Main  Instrument  Panel 
Display  is  shown.  From  there  he  can  progress 
to  other  displays  entitled  Left  Vertical  Console, 
Left  Console,  Right  Vertical  Console,  and  Right 
Console.  These  displays  depict  the  position  of 
switches  and  other  controls  on  the  indicated 
panels  and  consoles. 

If  the  instructor  selects  Map  Displays,  he 
can  go  to  a  series  of  graphic  displays  entitled 
Cross-Country,  Approaches,  Departures,  and  GCA. 

The  Cross-Country  Display  is  a  map  depicting  the 
radio  navigational  facilities  in  the  gaming  area 


and  showing  an  aircraft  symbol  and  track;  the  map 
coordinates  are  established  by  the  initial  condi¬ 
tions  set  entered  by  the  instructor.  The 
instructor  can  change  the  scale,  operate  Map  Slew 
and  Aircraft  Slew,  change  the  visual  scene,  ini¬ 
tiate  two  levels  of  declutter,  and  take  other 
actions  by  illuminating  appropriately-identified 
strike  areas  at  the  bottom  of  the  page.  He  can 
exit  only  to  the  previous  display. 

The  Approaches  and  Departures  Displays  con¬ 
tain  indexes  of  42  published  approaches  and  22 
standard  instrument  departures.  If  the  instructor 
selects  any  of  these,  a  depiction  of  the  DOD 
approach  or  departure  plate  is  shown. 

The  GCA/CCA  Display  shows  the  glide  slope 
and  course  line,  as  is  standard  for  this  type  of 
display.  This  display,  as  well  as  the  display  of 
individual  approaches  or  departures,  contains  a 
functional  control  capability  similar  to  that  on 
the  Cross-Country  Display,  although  it  is  tailored 
to  fit  the  individual  display.  For  example,  the 
GCA/CCA  display  does  not  have  Map  Slew,  Aircraft 
Slew,  scale-changing,  and  declutter. 

The  method  of  controlling  malfunctions  and 
the  display  of  emergency  (and  normal)  procedures 
is  intended,  again,  to  minimize  demands  on  the 
instructor.  The  instructor's  first  step  in 
activating  one  or  more  malfunctions  is  to  call 
up  the  Menu  Display  from  which  he  can  go  to  the 
Malfunctions  Display.  Next  he  selects  the  sub¬ 
display  for  the  category  of  malfunction  that  he 
desires;  the  categories  are  Flight  Control, 

Engines,  Fuel,  Hydraulic,  Electrical,  Comm/Nav, 
Display  Subsystems,  Aircraft  Systems,  and  EMI. 

The  display  that  he  selects  contains  a  list  of 
malfunctions  with  related  strike  areas,  one 
for  the  left  engine  and  one  for  the  right  when 
applicable  (the  display  for  the  engine  malfunc¬ 
tions  is  shown  in  Figure  7).  At  the  Lop  of  the 
page  are  a  list  of  three  "options"  and  four 
"events".  The  options  are  labeled  IMMEDIATE 
EFFECT,  REMOVE,  and  TIMED,  and  the  events  are 
FLAPS  UP,  FLAPS  DN,  GEAR  UP,  and  GEAR  DN .  By 
illuminating  with  the  light  pen  the  appropriate 
strike  area  the  instructor  can  activate  a  mal¬ 
function  immediately  or  in  the  future.  Future 
malfunctions  can  be  related  to  a  mission  clock 
time,  which  the  instructor  must  enter  with  the 
keypad,  or  to  one  of  the  four  events.  From  this 
display  the  instructor  can  exit  to  Lhe  previous 
or  next  display  in  the  same  category,  if  there  is 
more  than  one  available,  or  he  can  return  to  the 
Malfunctions  Display,  where  he  can  find  his  way 
through  a  similar  route  to  another  list  of  mal¬ 
functions  . 

When  the  instructor  calls  up  a  map  display 
after  entering  malfunctions,  he  can  observe  up  to 
ten  current  and  future  malfunctions  listed  at  the 
bottom  of  the  page,  in  an  area  called  the  All-Modes 
Area.  Strike  areas  are  provided  for  the  current 
malfunctions  so  that  the  instructor  can'  clear  them 
whenever  he  desires. 

Emergency  and  normal  procedures  are  displayed 
in  a  way  that  is  consistent  with  other  displays. 

The  Procedures  Display,  which  the  instructor 
obtains  via  the  Menu  Display,  enables  access  to 
four  sub-displays  containing  lists  of  procedures: 
Normal,  Ground  Emergency,  Takeoff  Emergency, 
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Figure  7.  Engine  Malfunctions  Display 
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Inflight  Emergency,  and  Landing  Emergency.  After 
the  instructor  selects  (with  the  light  pen  or 
keypad)  the  procedure  that  he  wants,  it  appears 
in  a  "ready"  status  on  a  divided  display 
entitled  the  Procedure  Monitor  Display  (see 
Figure  8).  The  instructor  can  place  it  in 
"active"  status  so  that  it  indicates  whether 
the  student  is  accomplishing  the  steps  in  the 
correct  sequence;  advance  the  steps,  if  there 
are  more  than  16  in  the  procedure,  so  that  the 
second  half  is  displayed;  retract  from  the 
second  half  to  the  first  half;  and  clear  the 
malfunction  related  to  the  procedure,  if  it 
is  an  emergency  procedure.  These  actions  are 
taken  by  illuminating  with  the  light  pen  the 
strike  areas  in  a  list  of  instructions  at  the 
bottom  of  each  procedure. 

Emergency  procedures  are  displayed  auto¬ 
matically,  also,  when  a  related  malfunction 
occurs . 

Additional  displays  include  those  used  to 
initiate  and  control  the  programs  entitled 
Dynamic  Replay,  Minute  Replay,  and  Print;  an 
extensive  series  of  displays  for  Programmed 
Missions;  a  display  for  controlling  and  monitor¬ 
ing  Parameter  Recording;  an  index  of  available 
demonstrations;  and  a  family  of  sub-displays 
originating  from  tb  Environment /Aircraft  Data 
Display  and  containing  the  parameters  for 
varying  the  weather,  visual  system  conditions, 
aircraft  configuration,  external  stores  and 
carrier  conditions,  and  for  recording  and 
displaying  landing  and  ejection  data.  Since  the 
F/A-18  OFT  has  generally  the  same  instructional 
programs  as  the  EA-6B  WST,  its  repertoire  of 
displays  is  equally  as  large. 

User  Experience 

The  F/A-18  OFT  will  be  used  initially  by 
VFA  125,  a  replacement  training  squadron  at 
Lemoore,  and  later  by  a  similar  organization 
at  Cecil.  Follow-on  trainers,  if  procured,  will 
be  used  by  operational  squadrons. 

The  first  trainer  is  currently  in  HSI 
(hardware/software  integration),  and  until 
Government  testing  counences  there  will  ba  no 
user  experience  to  draw  on  for  analysis  purposes. 
Operational  user  experience,  of  course,  will  not 
be  obtainable  until  after  final  acceptance, 
scheduled  for  early  1982.  At  present  it  is 
possible  only  to  estimate,  based  on  an  assessment 
of  the  trainers'  characteristics,  vhat  user 
opinions  will  be. 

There  is  no  reason  to  conclude  that  the 
users  will  not  be  satisfied  with  the  general 
features  of  the  IOS.  Navy  pilots  in  the  Fleet 
Project  Team  have  been  closely  involved  in  its 
development,  commencing  with  the  initial  mock-up. 
In  fact,  the  concept  of  emulating  the  F/A-13 
cockpit  originated  with  them.  Sperry  SECOR's 
initial  design,  presented  at  the  Mock-Up  Confer¬ 
ence,  envisioned  a  conventional  IOS  liberally 
equipped  with  push-button  switches  for  control  of 
functions  and  displays  and  featuring  nen-dipplay 
dependent  formats  with  input  codes  similar  to 
those  on  the  EA-6B  WST.  At  the  end  of  the 
conference  the  Government  concluded  that  the 
F/A-18  pilots  who  were  assigned  to  be  instructors 
on  the  trainer  would  resist  learning  how  to 


operate  a  sophisticated  IOS.  It  was  stated  that 
the  demands  of  flying  the  aircraft  and  operating 
its  advanced  systems  would  be  so  engrossing  that 
another  substantial  learning  task  would  be  unduly 
burdensome.  Sperry  SECOR  was  then  charged  with 
developing  the  cockpit-oriented  approach. 

There  are  some  questions  to  be  answered  when 
the  users  acquire  experience  with  the  IOS.  The 
first  is  whether  the  intended  use  of  the  light 
pen  will  be  more  successful  than  this  method  has 
proven  to  be  with  the  EA-6B  WST.  Sperry  SECOR 
anticipates  that  the  use  of  strike  areas,  plus 
faster  processing,  will  correct  the  previous 
problems,  which  appear  to  have  been  caused  pri¬ 
marily  by  attempting  to  illuminate  individual 
numerals  or  letters,  much  smaller  targets.  Pre¬ 
liminary  testing  of  the  light  pen  has  shown 
encouraging  results,  but  the  proof  will  be 
whether  the  instructors  and  device  operators, 
under  the  stress  of  a  fast-moving  training 
problem,  use  that  method  or  resort  to  the  back-up 
method,  the  keypad  and  line  numbers.  Use  of  the 
keypad  will  not  be  unusual  for  the  instructors 
because  they  will  have  experience  with  a  similar 
device  on  the  aircraft's  Up-Front  Control. 

The  second  question  is  whether  the  elaborate 
"cascading"  of  displays  will  result  in  too  much 
manipulation.  There  are  many  successive  indexes 
in  the  display  structure,  and  they  may  delay  the 
instructor's  finally  obtaining  the  display  that 
he  desires.  Also,  it  appears  that  there  are 
very  few  choices  available  in  the  exit  lines.  A 
more  liberal  use  of  options  would  possibly  enable 
the  instructor  to  find  short-cuts. 

Finally,  there  is  a  question  regarding  the 
function  of  the  device  operator.  Because  of 
the  design  of  the  IOS,  his  role  will  be  limited 
to  those  functions  that  can  be  exercised  through 
the  displays.  The  functions  accomplished  by  the 
switches  grouped  around  the  visual  monitor  are 
reserved  for  the  instructor — there  is  no  redundant 
capability  available  to  the  device  operator.  This 
approach  requires  that  instructors  be  uniformly 
well-trained;  if  one  is  not  capable  of  operating 
his  portion  of  the  instructor  station,  there  is 
no  way  for  the  device  operator  to  compensate 
Fortunately,  the  instructor's  controls  are 
relatively  simple,  and  this  situation  may  not 
become  a  problem. 


CONCLUSIONS 

It  is  impossible  to  state  conclusively  that 
on  ;  of  the  three  designs  is  the  "best"  or  the 
"most  efficient".  The  users  of  each  trainer  are 
generally  satisfied  with  their  own  IOS,  assuming 
that  the  opinion  of  the  F/A-18  users  will  be  as 
expected;  there  are  valid  reasons  for  having 
selected  each  of  the  designs  used;  and  there 
certainly  is  no  reason  to  conclude  that  one 
design  should  have  been  adopted  for  all  three 
trainers . 

The  input  codes  and  non-display  dependent 
formats  of  the  EA-6B  WST  seem  to  be  appropriate 
for  that  trainer,  in  view  of  its  many  instructional 
features  and  the  complexity  of  its  IOS.  Addition¬ 
ally,  the  use  of  function  keys  for  calling  up 
displays  and  for  controlling  operating  functions 
and  instructional  programs  serves  to  facilitate 
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Figure  8*  Procedure  Monitor  Display 


and  expedite  the  operations  involved.  On  the 
other  hand,  the  simpler  design  philosophy 
adopted  for  the  NCLT  suits  the  limited  mission 
of  that  trainer.  Finally,  the  cockpit-oriented 
IOS  of  the  F/A-18  OFT  can  be  considered  to  be 
successful  if  it  results  in  instructor  pilots 
wanting  to  become  trainer  instructors.  If  the 
light  pen  proves  not  to  be  preferred  by  instruc¬ 
tors,  the  back-up  method  is  merely  another 
version  of  the  input  method  used  on  the  NCLT — 
display-dependent  formats. 

Obviously,  improvements  can  be  made  at 
each  IOS.  For  the  EA-6B,  the  greatest  need 
appears  to  be  to  increase  the  capability  of 
the  device  operator.  This  can  be  accomplished, 
at  least  initially,  by  adding  the  mission  panels 
at  the  ECMO-1  Instructor  position;  further  design 
changes,  particularly  in  the  organization  of 
the  function  switches,  may  be  helpful.  In 
addition,  the  digital  readouts  on  the  Pilot's 
Instrument  Monitor  Display  should  be  changed 
to  a  pseudo-instrument  display;  and,  if  the 
strike  areas  on  the  F/A-18  OFT  displays  solve 
the  light  pen  problem,  the  EA-6B  WST  displays 
should  be  changed  accordingly. 

On  the  NCLT,  apparently  the  only  major 
improvement  needed  is  the  addition  of  a  function 
switch  to  start  the  parameter  recording 
procedure. 

Regarding  the  F/A-18  OFT,  it  is  too  early 
to  predict  what  improvements  will  be  needed. 

The  most  likely  candidate,  however,  is  the 
organization  of  the  displays,  particularly  the 
number  of  levels  of  indexes. 

The  principal  single  conclusion  that  can 
be  drawn  from  the  foregoing  analysis  of  IOS 
designs  is  that  it  is  very  difficult  to  design 
an  IOS  that  meets  the  operating  requirements  of 
a  modern  trainer  with  sophisticated  instructional 
features  and  yet  is  so  simple  that  it  can  be 
operated  by  an  instructor  pilot  who,  perhaps 
through  no  fault  of  his  own,  cannot  keep 
current  in  the  use  of  its  controls  and  display. 
The  F/A-18  OFT  is  an  attempt  to  solve  this 
dilemma  by  using  an  IOS  that  resembles  a  cockpit 
with  which  the  instructor  is  already  familiar. 
Another  solution  is  to  design  the  IOS  so  as  to 
maximize  the  role  of  a  device  operator,  who  can 
be  thoroughly  trained  and  kept  current,  and  to 
allow  the  instructor  to  concentrate  on  teaching 
and  monitoring  the  student. 


LIMITATIONS  OF  THE  PAPER 

The  only  investigative  method  used  in  the 
development  of  this  paper  has  been  interviews 
with  designers  and  users  of  the  trainers.  These 
contacts  have  been  very  productive,  but  subjec¬ 
tivity  is  always  a  factor  to  be  considered. 
Unfortunately,  it  has  not  been  possible  to  try 
to  develop  objective  measures  of  efficiency. 

Some  thought  has  been  devoted  to  the  subject. 
Possible  approaches  are  to  measure  the  time 
required  to  accomplish  representative  control 
functions  and  to  determine  the  associated  error 
rate.  Typical  functions  common  to  all  three 
trainers  are  activating  and  removing  malfunctions, 
modifying  environmental  parameters  such  as  sea 


state  and  wind  speed  and  direction,  modifying 
visual  system  parameters  such  as  ceiling  and 
visibility,  and  modifying  aircraft  parameters  such 
as  fuel  load  and  external  stores  configuration. 

An  additional  function  worth  evaluating  is  the 
control  of  instructional  programs  such  as  demon¬ 
strations  and  programmed  missions;  however,  all 
three  trainers  do  not  have  the  same  instructional 
capabilities  and  any  comparisons  in  this  area 
would  be  incomplete. 

An  approach  worth  noting  is  that  taken  in  a 
report  prepared  by  the  Boeing  Aerospace  Company 
for  the  Air  Force  Human  Resources  Laboratory 
entitled  "Instructor/Operator  Display  Evaluation 
Methods"  (1).  The  20  subjects  participating  in 
the  study  observed  flight  indications  on  two 
types  of  displays — digital  readouts  and  repeater 
instruments — and  were  required  to  answer  questions 
demonstrating  their  short-term  recall  of  the  data 
displayed.  The  use  of  digital  versus  analog 
readouts  is  not  an  issue  with  the  three  trainers 
discussed  in  this  paper — the  digital  instrument 
displays  of  the  EA-6B  WST  have  already  been  found 
wanting — but  the  question-and-answer  approach 
with  a  large  number  of  subjects  involved  would  be 
a  useful  technique  in  any  investigation, 

A  major  difficulty  in  making  objective  evalu¬ 
ations  of  the  three  IOS  designs  with  which  we  are 
concerned  is  that  the  process  of  obtaining  data 
would  require  considerable  effort  and  would  tend 
to  disrupt  the  normal  training  activities  at  the 
device  sites.  If  the  users  participate  in  the 
data-gathering  effort,  the  disruption  would  be 
major.  Measurement  of  elapsed  time  and  calcula¬ 
tion  of  error  rate  could  possibly  be  done  with  a 
computer  program  that  would  operate  unobtrusively 
while  training  is  in  progress,  but  such  an  approach 
would  be  expensive. 

It  is  anticipated,  in  this  case,  that  there 
would  be  a  question  regarding  the  significance  of 
observed  differences.  It  can  be  concluded,  from 
casual  observation,  that  the  time  to  make  an 
average  entry  with  a  keyboard  will  not  be  more 
than  two  or  three  seconds  more  than  with  a  light 
pen,  assuming  that  the  latter  is  operating 
optimally.  Such  differences  are  not  important, 
particularly  when  the  operations  can  be  performed 
by  the  device  operator  while  the  instructor  is 
concentrating  on  observing  the  student's  perform¬ 
ance  . 

In  any  evaluation  of  the  efficiency  of 
display-dependent  formats,  the  time  to  call  up 
the  display  should  be  added  to  the  time  to  make 
the  entry.  If  the  displays  are  designed  to  use  a 
maximum  number  of  lines  per  page,  there  will  be 
less  need  for  display  manipulation,  and  this  factor 
will  have  a  lessened  impact  on  efficiency.  ThuB , 
display  design  would  be  a  variable  that  would 
have  to  be  considered  in  the  evaluation. 

In  spj"-  of  the  limitations  of  the  interview 
method,  thi  pproach  is  probably  the  one  best 
suited  to  the  problem  of  evaluating  an  instructor 
station.  Additionally,  a  detailed  study  and 
careful  analysis  of  the  design  features  involved, 
accomplished  conscientiously  by  the  investigator, 
are  necessary  elements  of  the  evaluation.  This 
has  been  the  case  in  the  development  of  this 
paper. 
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ABSTRACT 

The  training  of  combat  vehicle  crews  and  com  manders  via  Interactive  Computer  Graphics  is  a  feasible 
and  effective  method  to  maintain  readines  against  the  existing  threat  of  Nuclear  Biological -Chemical  (NBC) 
warfare.  Data  and  computer  programs  have  been  generated  which  provide  real  world  chemical  battlefield 
scenarios.  The  system  user  encounters  different  battlefield  scenarios  and  can  select  various  defensive 
responses  (e.q.  initiate  overpressure).  Dosages  encountered  and  casualties  (for  unprotected  crew  members) 
suffered  in  the  chemical  battlefield  simulation  are  tabulated. 


BACKGROUND 

Nuclear,  Biological,  and  Chemical  (NBC)  Threat 

Little  attention  to  Chemical/  Biological  (CB)  war¬ 
fare  threat  has  been  exhibited  in  the  last  decade.  This 
lack  of  concern  was  due  mainly  to  the  assumption  that 
any  Chemical/Biological  action  would  escalate  directly 
into  a  full  scale  nuclear  war.  Current  inform ati.cn  indi¬ 
cates  that  the  Soviet  Union  is  well  prepared  and  equipped 
to  fight  a  small  scale  tactical  NBC  offensive.  This  fact 
was  proclaimed  in  an  overview  statement  of  the  US 
military  pasture  far  FY82  by  the  chairman  of  the  Joint 
Chiefs  of  Staff: 

"The  Soviet  Union  is  the  best  prepared  nation 
to  conduct  offensive  chemical  warfare 
operations.  DoctrinaUy  chemical  warfare 
capabilities  are  tied  to  employment  ir.  a 
nuclear  war.  However,  the  evidence  of  a 
Soviet  use  of  chemical  weapons  in  a  non¬ 
nuclear  scenario  is  not  clear,  but  the  possi¬ 
bility  cannot  be  discounted.  The  Soviets  and 
their  surrogates  may  have  used  lethal  and 
nonlethal  chemical  agents  in  Afghanistan, 

Laos,  and  Kampuehan."  (1) 

It  is  contended  that  after  the  chemical  release 
decision  is  made,  autiiority  for  use  would  bo  delegated 
down  to  the  Soviet  divisional  commander  level  where 
chemical  weapons  could  be  used  freely  as  a  normal  com¬ 
pliment  to  conventional  weapons.  The  Soviets  employ 
50,000  to  80,000  chemical  troops  whose  mission  it  is  to 
support  (technically  and  in  training)  each  military  divi¬ 
sion. 

Sources  estimate  fifteen  percent  cf  an  Soviet 
warheads  contain  chemical  agents.  Many  Soviet  combat 
vehicles  employ  collective  protection  and  troop  training 
in  a  CB  environment  is  extend. ve,  including  expoang 
troops  to  actual  toxic  chemical  agents.  Soviet  prepara¬ 
tion  for  NBC  warfare  is  indeed  awesome. 


Need  for  United  States  Chemical  Warfare  Readiness 

It  is  clear  the  Soviets  have  the  capability  to  fight  a 
tactical  NBC  war.  US  preparedness  to  fight  in  a  NBC 


environment  is  the  best  deterrent  to  the  Soviets  initiating 
a  chemical  attack.  Adequate  readiness  requires  both 
development/fielding  of  NBC  protected  hardware  and  the 
impxovementy'deveiopment  of  training  procedures  to 
prepare  troops  for  operations  in  NBC  suroundinga 


INTRODUCTION 

Need  far  Combat  Vehicle  Crew  Training 

The  NBC  environment  is  a  mark*  change  from  the 
normal  battlefield.  Possibly  more  devastating  than  toxic 
agent  casualties  will  be  the  physiological  and  psycho¬ 
logical  effects,  impaired  vision,  impaired  communi¬ 
cation,  restricted  tactile  deftness,  and  thermal  loading, 
along  with  latent  claustrophobia  and  fear  will  make 
extremely  difficult  the  rational  thinking  required  for  the 
combat  vehicle  crew  on  a  NBC  battlefield.  To  counter 
this  problem  combat  vehicle  crew  and  commanders  must 
be  trained  not  only  to  survive  a  CB  attack,  but  to  react 
correctly  to  all  expected  situations.  Since  conducting 
practice  maneuvers  in  a  NBC  environment  is  unrealistic 
due  to  limitations  impxsed  by  the  Surgeon  General,  a 
supplemental  training  method  is  required  that  will 
properly  indoctrinate  these  com  belt  vehicle  crews  to  the 
requirements  and  limitations  intrinsic  to  NBC  warfare. 
This  paper  proposes  that  an  Interactive  Computer 
Graphical  Training  Aid  can  provide  such  a  system. 


THE  INTERACTIVE  GRAPHICAL  TRAINING  AID 
Development 

The  Interactive  Graphical  Training  Aide,  an  exten¬ 
sion  of  previous  chemical  battlefield  computer 
simulation,  predicts  mathematically  the  chemical  threat 
to  armored  combat  vehicle  crews  (tanks,  armored  person¬ 
nel  carriers,  etc.).  This  model,  designated  CHEMWAM 
(Chemical  Vehicle  Vulnerability  Analysis  Model),  was 
initially  developed  to  be  an  aid  to  combat  vehicle  col¬ 
lective  protection  develop  m  ental  engineers  and  as  a  tool 
to  assess  current  chemical  and  bioLogical  threat  readi¬ 
ness.  (2)  Use  of  CHEMWAM  for  a  graphical  computer 
training  aid  was  later  pcopiosed  by  the  authors.  To 
accomplish  this  last  proposal  software  for  an  interactive 
graphical  Emulation  of  a  chemical  battlefield  was 
designed. 


175 


U9e  of  the  Interactive  Computer  Graphical  Training  Aid 


System  users  will  require  no  knowledge  of  either 
computer  programming  cr  lanquage.  AH  necessary  para¬ 
meters  are  queried  by  the  computer.  The  system  user 
algo  has  the  option  to  request  a  more  detailed  explanation 
of  each  parameter  required.  The  system  U9er  selects  the 
battlefield  scenarios  (stipulating  weather,  temperature, 
agent,  mmiticn  type,  number  of  munitions),  and  the 
speed  and  operational  configuration  of  the  vehicles.  The 
computer  then  initiates  the  simulation.  The  system  user 
then  chooses  the  output  deaxed.  Among  these  outputs 
are  graphical  representations  of  the  battlefield  (vehicle 
positions  and  agent  concentration  gradients)  and  listings 
of  crew  dosages  and  vehicle  internal  concentrations. 
Histories  of  the  vehicle  internal  concentration  and  crew 
dosages  can  also  be  tabulated.  From  these  varied  outputs 
the  system  user  can  assess  the  requirements  and  limita¬ 
tions  inherent  in  fighting  on  a  chemical  battlefield.  It  is 
anticipated  that  this  type  of  training  can  prepare  the 
combat  vehicle  crew  to  make  appropriate  response  to 
NBC  situations  as  they  arise.  Furthermore  the  ease  of 
administrating  this  training  enables  one  to  train  often  and 
thereby  reinforce  proper  reactions  in  given  situations. 
However,  to  minimize  physiological  stress  field  training 
m  ust  supple  m  ent  this  training. 


Mode  of  fmcticn 

By  inputing  selected  parameters  into  the  mathe¬ 
matics  model,  CHRMWAM  generated  battlefield 
conditions  at  various  times  after  attack  cam  be 
observed.  This  data  can  be  listed  numerically  or  repre¬ 
sented  graphically  on  a  battlefield.  CHEMWAM, 
programmed  in  FORTRAN,  is  executable  from  the 
Chemical  Systems  Laboratory  UNIVAC  1108  computer. 


Mathematical  Model  (CHEMWAM) 

CHEMWAM  is  an  integration  of  four  mathematical 
models:  1.  NUSSE  -  Non  Uniform  Simple  Surface  Evap¬ 
oration  Model,  2.  PARACOMPT  -  Parametric  Analysis  Of 
Respiratory  Agents  Considering  Operations,  Motivation, 
Protection  And  Time  ModeL,  3.  Data  Analysis  Model, 
and  4.  Vehicle  Track  ModeL 

These  four  models  interact  to  generate  effects  from 
the  time  the  attack  begins  until  the  vaocr  cloud  has 
disipated.  The  generated  effects  include:  concentration 
levels  at  each  pdnt  on  the  battlefield,  vehicle  movement, 
vehicle  interior  concentrations,  and  crew  dcsages.  A 
description  of  each  model  is  presented. (3) 


NUSSE.  The  NUSSE  model  simulates  the  delivery  of 
a  chemical  agent  from  a  ringle  munition  onto  a  target 
area.  Agent  dissemination  can  be  either  an  air  burst 
(breakup  of  a  liquid  mare  along  an  inclined  line)  cr  a 
surface  burst  (a  point  source).  Given  an  air  burst,  NUSSE 
simulates  the  fall  of  liquid  through  the  air  and  liquid 
evaporation  during  its  fall  to  the  surface  disposition 
pattern.  Fcr  the  surface  burst,  only  the  disposition 
pattern  is  calculated.  Then  dosages  are  computed  fee 
each  point  cn  the  target  area  art  various  times  after  the 
agent  contamination.  This  series  of  dosage  grids  presents 
an  accurate  history  of  the  dispertion  of  a  chemical  agent 
from  a  single  munition.  The  NUSSE  model  is  itsalf  an 
integration  of  three  separate  models  developed  at 
Chemical  Systems  Laboratory  in  1978.  A  pictorial 
representation  of  the  NUSSE  model  is  shown. 


PARACOMPT.  The  PARACOMPT  Model  takes  the 
effects  of  a  single  munition  as  computed  by  NUSSE  and 
aggregates  the  effects  fcr  a  multipile  weapons  attack. 
W  eapon  delivery  errors  and  firing  rates  are  included.  The 
resulting  output  represents  the  effects  ol  the  entire 
multiple  munition  chemical  attack  in  a  time  series  of 
dosage  grids  analogous  to  those  produced  by  NUSSE  for  a 
single  m  unition.  This  is  shown  schem  atically  below: 


Input:  Time  aeries  of  secondary  evaporation  dosage 
grids  are  produced  by  the  NUSSE  ModeL 

Output:  Time  series  of  aggregate  dosage  grids, 
each  such  grid  reflecting  the  total  dosage  resulting  from 
the  set  of  munitions  analyzed,  at  each  point  on  the  grid, 
at  the  paint  in  time  with  which  the  particular  grid  is 
associated. 


The  PARACOMPT  Model  was  developed  in  1963  at 
Chemical  Systems  Laboratory.  Modifications  were  made 
to  accept  the  input  from  NUSSE  and  present  the  output  in 
a  format  acceptable  to  the  third  model  in  this  system.  It 
should  be  noted  that  a  specific  target  has  not  been 
addressed  in  the  system  of  models  yet.  Chemical  agent 
effects  have  been  computed  and  aggregated  with  refer¬ 
ence  to  a  specific  grid. 

Data  Analysis  ModeL  This  model  was  developed 
based  on  test  data  from  combat  vehicle  simulant  chal¬ 
lenge  testing.  The  following  concept  fcr  ingress  and 
eg r ere  of  chemical  agents  for  combat  vehicles  is  a  math¬ 
ematical  representation  of  observed  test  results: 
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cx  External  ConcentratLcn 
Internal  Concentration 
V  internal  Volume 

Capacity  erf  Channels  for  Ingreas 
of  Agent 

S2  Capacity  of  Channels  for  Egress 

of  Agent 

F^  Flow  of  Agent  into  Vehicle 
F2  Flow  of  Agent  out  of  Vehicle 

Assumptions  concerning  the  above  process: 


(U  Agent  ingress  is  the  product  of  the 
difference  between  internal  and  external  vehicle  concen¬ 
tration  and  vehicle  leakage  inward. 

(2)  Vehicle  internal  concentration  changes  as 
a  ratio  of  agent  net  flow  to  internal  volume  of  air. 

(3)  Agent  egress  is  the  product  of  internal 
vehicle  concentration  and  vehicle  leakage  outward. 


The  assumption  as  equations  are: 


and 


F1  =  S1  <Cx  “  Ci) 


C  =  ±  <Fl 
'*•  V 


F2> 


F2  =  S2Ci 


Ci 


Sl  +  s2 


Ci  + 


The  equations  jure  the  basis  for  a  simple  first  order 
model  representing  chemical  vapor  ingress/egress  for  a 
combat  vehicle.  Although  the  proce®  of  agent  ingress' 
egress  is  continuous,  this  process  uses  a  discrete  formu¬ 
lation.  The  two  levels  of  this  model  are  shown  below: 


Xktl  =  f  *  Xk 
f  =  e-a  ^ 


whereat  =  tfc+1  -  tk 


INTERNAL  CHALLENGE 


Xk+1  "  f  *  Xk  +  9 
f  -  e-aAt 

g  -  b/a  (l-e"aAt) 


U 


k+1 


EXTERNAL  CHALLENGE 
*  Internal  concentration  at  time  k,  X  (t)  for  t«k 
Uk  »  External  concentration  at  time  k,  U  (t)  for  t*k 
f,  g  =  Transition  matrics 

The  regression  model 

Xi  “  F  xi-1  +  g  ui  becomes  the  algorithm  fer 
predicting  agent  ingress. 


Vehicle  Track  Model 

The  of  the  four  major  models  of  the  system, 
this  model  calculates  the  internal  vehicle  dosage  history 
as  the  vehicle  is  passing  through  a  contaminated  area. 
The  time  histories  can  be  calculated  for  a  group  of 
twenty-five  combat  vehicles.  The  combat  vehicles 
against  which  chemical  effects  are  to  be  asBe®ed  are 
described  in  terms  erf  location,  direction  erf  movement, 
speed  and  configuration  (ventilation  parameters). 
Vehicles  that  move  and  react  as  a  group  can  be  described 
as  a  group,  however,  individual  vehicle  dosage  histories 
will  still  be  calculated  as  the  vehicle  follows  its  own 
route  aero®  the  battlefield.  Vehicle  operational  config¬ 
uration  (hatches  open,  blowers  on/crff)  can  be  varied  as 
vehicle  moves  through  the  PARA  CO  MPT  grid. 


FUTURE 

Planned  development  of  this  system  at  Chemical 
Systems  Laboratory  will  continue  until  the  CHEMWAM 
program  has  the  capability  to  simulate  a  chemical  battle¬ 
field  for  a  battalion  level  force. 

It  is  suggested  that  simulations  employing  similar 
techniques  as  CHEMWAM  be  developed  to  better  repre¬ 
sent  the  actual  tank  environment  on  a  total  NBC  battle¬ 
field.  This  would  provide  a  unique  stimulus-iesponse 
reinforcement  for  the  system  user  and  further  insure 
proper  reactions  to  situations  encountered  on  a  NBC 
battlefield. 


CONCLUSION 

By  affording  combat  vehicles  crews  and 
commanders  the  opportunity  to  actually  see  (if  only 
graphically)  the  results  of  many  possible  NBC  batflefidd 
scenarios,  they  can  determine  what  their  decisions  will 
have  on  the  combat  vehicles  capability  to  effectively, 
fight  in  a  chemical  battle.  Additionally  the  training 
system  is  practical  and  inexpendve  to  operate,  thereby 
making  continued  retraining  feasible.  Continued  expo¬ 
sure  to  chemical  battlefield  scenarios  will  better  ensure 
that  through  both  field  exercises  and  simulation  training 
the  US  combat  ehicle  fleet  will  be  effective  on  a  NBC 
contaminated  battlefield. 
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ABSTRACT 

Training  Effectiveness,  both  in  terms  of  measurement  and  prediction,  has  become  an  issue  of 
increasing  concern  to  training  system  users,  acquisition  managers,  and  contractors.  The  analytic 
tools  available  to  address  this  issue  have  generally  come  from  the  realm  of  Human  Factors  and 
have  tended  to  focus  on  individual  training  devices  rather  than  total  training  systems.  This 
paper  describes  a  training  system  effectiveness  model  that  addresses  all  elements  of  a  Jet 
pilot  training  program  (academics,  simulators  and  aircraft)  which  was  developed  through  the 
application  of  Operations  Analysis  and  Instructional  System  Development  techniques.  The 
processes  used  to  identify  and  validate  characteristics  which  drive  training  effectiveness 
are  described,  as  are  the  methods  used  to  relate  these  characteristics  to  training  objec¬ 
tives.  The  techniques  used  to  quantify  the  training  value  associated  with  various  design 
options  are  presented  along  with  a  discussion  of  how  the  model  was  actually  used  during  the 
training  system  design  process. 


INTRODUCTION 

Within  the  training  community  the  concept  of 
training  effectiveness  has  emerged  as  a  major  issue 
influencing  virtually  every  aspect  of  the  training 
equipment  procurement  cycle  -  from  the  initial  user 
definition  of  need;  through  the  specification  pro¬ 
cess;  and  into  design,  development,  test  and  ac¬ 
ceptance  of  the  equipment.  The  reason  is  clear: 

The  continuing  proliferation  of  training  devices 
in  terms  of  numbers,  complexity  and  cost  has 
reached  a  point  where  "mistakes"  in  the  form  of  de¬ 
vices  and  systems  which  are  ineffective  and/or  in¬ 
efficient  simply  cannot  be  afforded.  Military 
training  equipment  procurement  agencies  have  re¬ 
sponded  by  incorporating  training  effectiveness 
clauses  in  recent  device  specifications  and  by  in¬ 
cluding  training  effectiveness  "tests"  as  part  of 
the  government  acceptance  process .  Our  experience 
with  these  new  requirements  began  in  Mid  1978  with 
the  Navy's  A6  TRAM  Detection  and  Ranging  Set 
maintenance  trainer.  Here,  and  for  the  first  time 
in  our  experience,  the  contractor  was  required  to 
deliver  a  plan,  including  test  objectives  and 
criteria,  for  the  empirical  demonstration  of  im¬ 
proved  student  performance  following  training  with 
the  device.  Satisfactory  completion  of  this  demon¬ 
stration  was  required  prior  to  Navy  acceptance  of 
the  trainer. 

A  similar  and  more  extensive  demonstration/ 
evaluation  is  planned  as  part  of  the  Arny  Main¬ 
tenance  Training  and  Evaluation  Simulation  System 
(AMTESS)  Program,  under  the  cognizance  of  PM  TRADE. 
The  concept  definition  phase  of  this  program  in¬ 
cluded  the  specification,  by  the  contractors,  of 
programs  of  instruction  to  be  used  with  the  trainer 
during  an  Army  evaluation  program.  These  programs 
are  currently  under  contractor  development  and  will 
be  delivered  with  the  prototype  trainers  in  early 
1982.  The  results  of  the  Army's  assessment  of  the 
relative  training  effectiveness  of  the  two  compe¬ 
ting  AMTESS  designs  will  be  a  major  factor  in  de¬ 
termining  the  future  of  the  program. 

Even  more  extensive  training  effectiveness  re¬ 
quirements  have  been  incorporated  by  the  Army  in 
the  trainer  programs  for  the  U.  S.  Roland  missile 
system.  For  ootli  the  Roland  Operation  and  Main¬ 
tenance  Trainers  contractors  are  being  required  to 
implement  formal  training  effectiveness  program 


plans  as  well,  as  supporting  the  Army  during  Train¬ 
ing  Effectiveness  Potential  Tests  to  be  conducted 
as  part  of  the  trainer  acceptance  process.  The 
planning  and  impiementationof  both  activities  are 
documented  in  formal  plans  submitted  in  response 
to  specific  contract  data  requirements. 

All  of  the  above  training  effectiveness  ef¬ 
forts  have  one  important  common  characteristic: 

They  focus  on  a  single  training  device  rather 
than  a  total  training  system.  The  responsibility, 
therefore,  for  effectively  integrating  the  train¬ 
ing  device  within  the  context  of  the  total  train¬ 
ing  system  (classroom,  actual  equipment,  etc.)  re¬ 
mains  principly  with  the  user. 

The  Navy's  Undergraduate  Jet  Flight  Training 
System,  VTXTS,  marks  a  major  departure  from  this 
approach.  This  program  encompasses  the  simul¬ 
taneous  development  of  all  four  major  elements  of 
flight  training:  academics,  simulators,  aircraft 
and  a  training  management  system.  The  Alternative 
System  Exploration  (ASE)  Study  phase  of  the  VTXTS, 
which  was  concluded  in  March  of  this  year,  in¬ 
cluded  a  requirement  to  include  consideration  of 
training  effectiveness  throughout  the  concept  de¬ 
finition  process  for  all  constituent  system  ele¬ 
ments  .  Contractors  were  specifically  required  to 
balance  the  training  effectiveness  of  each  can¬ 
didate  system  element  design  concept,  with  their 
anticipated  Life  Cycle  Cost.  The  training  ef¬ 
fectiveness  model  described  in  this  paper  was  de¬ 
veloped  as  a  tool  to  assist  the  Grumman/Beech/ 
Link  Team  in  addressing  these  requirements  for  our 
entry  into  the  VTXTS  competition,  called  System 
730.  The  model  was  developed  and  implemented 
through  the  joint  efforts  of  personnel  from  both 
our  Operations  Analysis  and  Instructional  Systems 
Development  groups . 

Evolution  of  Our  Approach 

Based  on  our  own  training  system  experience 
we-  knew  of  no  overall  approach  to  Training  System 
Effectiveness.  Indeed,  no  agreement  existed  as 
to  the  definition  of  the  term.  In  terms  of  the 
specific  issue  of  jet  pilot  training  we  were 
familiar  with  the  work  of  Bazzocchi  on  the  cost 
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effectiveness  of  trainer  aircraft,  (l)  Bazzocchi 
assumed  that  the  teaching  effectiveness  of  a  train¬ 
ing  aircraft  can  be  correlated  to  a  certain  number 
of  important  characteristics  related  to  performance 
and  equipment.  He  then  interrogated  expert  pilots 
to  rate  the  importance  of  each  characteristic  vis- 
a-vis  the  mission  listed.  Bazzocchi  concentrated 
only  on  the  aircraft,  not  the  entire  training 
system.  Moreover,  his  selection  and  definition  of 
characteristics  seemed  to  focus  more  on  the  per¬ 
formance  of  the  airplane  rather  than  on  its  role  as 
a  trainer.  Nevertheless,  the  concept  of  polling 
experts  offered  a  quantitative  technique  for  eval¬ 
uating  effectiveness  which  looked  attractive,  es¬ 
pecially  since  this  type  of  decision  theory  had 
been  used  successfully  by  the  operations  analysis 
members  of  the  VTXTS  team.  (2)  (3)  Thus  we  set 
out  to  develop  an  analytical  tool  by  which  we 
could  poll  and  evaluate  the  opinions  of  "experts" 
and  quantify,  in  a  relative  sense,  those  at¬ 
tributes  of  a  training  system  which  make  it  more 
effective.  We  emphasized  the  system,  and  sought 
to  go  beyond  what  Bazzocchi  did.  A  system  is  more 
than  a  training  aircraft.  Hence  we  included  in 
our  model  not  only  proposed  aircraft  designs  but 
also  the  simulators  and  academic  programs  and 
equipment  which  will  ultimately  make  up  the  system. 

THE  M5DEL 

The  model  is  described  by  the  logic  diagram  of 
Fig.  1.  The  essential  operations  involve: 

o  An  "importance  rating"  relating  system 
characteristics  to  the  87  Terminal 
Learning  Objectives  (TLOs)  provided  by 
the  Navy  for  the  VTXTS 

o  A  "criticality  factor"  which  weighs  the 
significance  of  each  of  these  learning 
objectives 

o  A  "system  amplification  factor"  which 
compares  an  alternative  system  to  a 
baseline. 


Training  System  Effectiveness 

Logic  diagram  shows  flow  of  analysis. 


FIGURE  OF  MERIT  -  E£A|j  it  Vj 
WHERE: 

Ajj:  UTILITY  RATING  FACTOR -Rij  1  F| 

Vj:  SYSTEM  AMPLIFICATION  FACTOR  (VALUE  RATIO  OF 
jtli  CHARACTERISTIC) 

Rij:  EXPERT  RATING  OF  TLO/CHARACTERISTIC  INTERFACE 
Fj:  CRITICALITY  FACTOR 
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From  these  three  assessments  a  relative  figure 
of  merit  (FOM),  or  relative  effectiveness  is 
generated. 

Selection  of  System  Characteristics 

The  core  of  the  approach  is  a  matrix  relating 
system  characteristics  to  learning  objectives . 

The  purpose  of  the  matrix  is  to  provide  a  format 
whereby  Navy  SM5s  could  rate  the  importance  of  a 
selected  group  of  aircraft,  simulator  and  academic 
characteristics  against  the  learning  objectives. 
Those  attributes  of  a  training  system  which  make 
it  more  effective  are  identified  and  quantified 
through  the  evaluation  of  polled  "expert"  opinion. 

The  chosen  characteristics  are  not  neces¬ 
sarily  simple  performance  measurements  but,  rather, 
the  salient  features  of  a  system  for  ensuring  ef¬ 
fective  training,  The  final  set  of  characteristics 
was  selected  through  discussion  with  the  Grumman 
team  and  with  Navy  SMEs .  This  resulted  in  a 
characteristic  set  which  was  comprehensive,  but 
not  too  lengthy.  An  overly  rigourous  list  would 
enlarge  the  matrix  and  increase  the  time  required 
for  the  ratings .  For  similar  reasons ,  the  set  of 
learning  objectives  used  waa.  not  the  87  TLOs,  but 
the  set  of  13  learning  stages  (for  aircraft  and 


Figure  1 


simulators)  and  eight  courses  (for  academics). 

The  final  matrices  employed  in  the  ASE  are  shown 
in  Fig.  2  and  3. 

Matrix  Rating  Procedure 

The  SMEs  were  asked  to  rate  each  of  the  boxes 
of  the  matrices  on  a  scale  of  0  to  3,  as  follows: 

o  0  =  not  applicable 

o  1  =  desirable 

o  2  =  important 

o  3  -  essential 

Thus,  if  a  particular  characteristic  was  absolutely 
necessary  for  attaining  the  objectives  of  a  learn¬ 
ing  stage,  a  three  would  be  inserted  in  that  box. 

If  it  had  no  bearing  at  all,  then  a  zero  would  be 
the  appropriate  entry,  and  so  on.  Many  in-house 
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VTXTS  Rating  Matrix  (Aircraft  and  Simulators) 

Mean  importance  ratings  of  82  SMEs. 
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VTXTS  Rating  Matrix  (Academics) 

Mean  importance  ratings  ot  82  SMEs. 

'V  Academic 

Syilam 

'v  Charatctariatics 

Laaming 

Stages 

m 

h 

Ii 

ij, 

if 

ii 

Flight  Support 

2.08 

1.72 

1.86 

2.06 

2.10 

Aviation  Student  Info 

1.23 

1.30 

1.4?. 

1.78 

1.86 

Engineering 

2.14 

1.78 

2.02 

1.70 

1.90 

Aerodynamics 

2.12 

1.70 

2,04 

1  90 

1.78 

Meteorology 

1.54 

1.40 

1.72 

1,58 

1.62 

Flight  Rules  &  Reg 

1.61 

1.42 

1.54 

1.70 

2.16 

IFR  Nav 

1.98 

1.58 

1.84 

1.92 

2.08 

Oper  Nav 

1.98 

1.64 

1.88 

2.10 

2.02 

MEAN  RATINGS 

1.84 

1.66 

1.79 

1.84 

1.94 

3:  Essential 

2:  Important 

1:  Desirable 

0:  Not  Applicable 

trial  runs  of  the  model  showed  that  the  scale  of¬ 
fered  a  wide  enough  spread  to  allow  for  distin¬ 
guishing  between  ratings.  It  was  compact  enough 
so  that  results  were  not  haphazard  considering  the 
sample  size. 

Flight  Instructors  and  academic  experts  were 
polled  at  four  Naval  Air  Stations.  In  addition, 
officers  at  CNATRA  participated  in  the  ratings . 

In  all,  82  SMEs  filled  out  the  matrices.  The 
arithmetic  means  of  their  ratings  are  the  numbers 
entered  in  each  cell  of  Fig.  2  and  3.  The  numbers 
at  the  bottom  of  each  column  are  the  means  for  each 
characteristic.  The  value  of  each  of  these  numbers 
dictates  the  importance  of  that  characteristic  to 
the  entire  learning  process.  They  are  the  basis 
for  the  bar  chart  in  Fig.  4  which  vividly  portrays 
the  expert  assessment  of  the  Navy  SM3s . 

importance  Ratings  -  For  the  aircraft,  there 
is  no  doubt  that  "student-instructor  interface" 
was  the  most  important  characteristic.  The  SM?s 
clearly  indicated  that  the  ability  of  the  in¬ 
structor  to  demonstrate  flight  maneuvers,  com¬ 
municate  wiuh  and  observe  the  student,  act  in  a 
safety  role,  and  in  general,  to  provide  close 
supervision  and  control  are  of  prime  importance. 
Figure  2  shows  that  "student -instructor  interface" 
not  only  has  the  highest  overall  rating,  but  in 
every  learning  stage  its  significance  is  domin¬ 
ating. 


Figure  3 


Two  other  characteristics  which  rate  highly 
are  "student-environment  interface"  and 
"stability  index".  The  first  of  these  deals  with 
human  factor  issues  within  the  cockpit. 
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was  placed  on  improvement  in  cockpit  design,  which 
resulted  in  improved  visibility,  communication, 
supervision  and  control. 

Aircraft  performance  was  not  overlooked. 
Despite  the  low  overall  ratings,  there  are  areas 
where  the  Navy  SMEs  deemed  performance  character¬ 
istics  of  importance.  For  example,  maximum  speed 
and  maximum  rate  of  climb  rank  respectively,  8  and 
7,  in  overall  importance  of  aircraft  character¬ 
istics,  rather  low  evaluations.  Yet,  for  the 
learning  stage,  "Air  Combat  Maneuvers",  both  of 
these  characteristics  were  rated  almost  essential 
by  the  SMEs.  This  type  of  infornation  served  as 
a  guide  to  the  designers  luring  this  study.  As 
refinements  are  incorporated,  the  dialogue  between 
system  designers  and  ISD/operations  analysts  con¬ 
tinues  to  encourage  the  development  of  a  more 
effective  trait.ing  system. 


Ratings  for  Simulators  -  The  ratings  for 
simulators  once  again  show  the  SMEs'  concern  for 
safety  and  communication  with  the  student.  The 
leading  characteristic  in  overall  rating,  and 
high  for  all  learning  stages,  is  "emergency  capa¬ 
bility".  This  is  a  measure  of  the  number  of 
possible  aircraft  malfunctions  and  the  type  of 
malfunctions  that  are  built  into  the  simulator 
system. 

The  next  most  important  characteristic  is 
"fidelity  to  aircraft  performance  and  stability". 
This  characteristic  also  rates  highly  across  all 
learning  stages,  and  would  seem  appropriate  given 
the  basic  purpose  of  simulator  training. 

"Instructor  console  interface"  follows  closely 
behind.  This,  again,  relates  to  the  communication 
between  instructor  and  student  and  the  ability 
of  the  instructor  to  monitor,  augment,  observe, 
and  control  the  activities  within  the  simulator. 

It  also  rates  well  across  most  of  the  learning 
stages. 

There  is  a  drop  in  characteristic  rating  after 
this.  "Visual  field  of  view",  "motion  cueing", 
and  "visual  characteristics"  have  evaluations  that 
are  somewhat  lower.  These  figures  can  be  used  as 
an  overall  indicator  of  relative  figure  of  merit 
(FOM)  for  comparing  systems.  However,  if  we  take 
a  closer  look  at  "visual  field  of  view"  and 
"visual  characteristics",  we  see  that  they  are 
rated  extremely  low  with  regard  to  learning  stages 
involved  with  instrumentation  and  airways  navi¬ 
gation.  For  the  learning  stages  involved  with 
weapons,  gunner,  ACM,  and  Carrier  Qual  they  rate 
quite  high. 

There  are  four  types  of  simulators  employed 
in  the  system.  Not  all  are  used  in  every  learn¬ 
ing  stage.  For  example,  ACM  is  accomplished  in 
the  "dome"  of  TFT  (Tactical  Flight  Trainer).  Here 
field  of  view  (FOv)  is  obviously  of  importance. 

It  would  be  an  erroneous  conclusion  if  FOV  would 
be  treated  as  unimportant  in  the  design  of  simu¬ 
lators  simply  because  of  its  overall  rating.  It 
is  important  for  the  ACM  stage  and  the  ratings 
show  it  as  so. 

These  observations  in  no  way  diminish  the 
importance  of  "emergency  capability"  or  "fidelity" 
to  the  overall  training  process.  They  rate 
highly  across  the  board.  It  does  accentuate  that 
care  must  be  used  in  applying  the  data. 

design  during  the  VTXTS  ASE  Study.  Emphasis 


VTXTS  Mean  Characteristic  Ratings 

Relative  importance  ot  system  characteristics,  based  on  composite! 
judgment  of  82  USN  training  command  SMEs. 


NOT  APPLICABLE 
0 


OESIRABLE 


IMPORTANT 

2 


SIMULATORS 


ACADEMICS 


R81-0001-026<HP)B 


Figure  4 


Stability  index  involves  the  ability  of  the  air¬ 
craft  to  fly  honestly  and  predictably,  with 
controllability  during  stalls  and  slow  flight, 
and  positively  controlled  entry  into  and  easy  re¬ 
covery  from  spins. 

Following  these  three  characteristics  in  im¬ 
portance  are  a  set  of  four  which  are  grouped 
closely  together;  important,  but  not  nearly  so  as 
the  first  three.  Beyond  these,  and  trailing  far 
behind  in  importance  are  "maximum  speed"  and 
"approach  speed". 

Some  overall  conclusions  are  drawn  from 
these  ratings.  The  SMEs  feel  that  for  the  train¬ 
ing  process,  an  aircraft  is  required  which 
stresses  human  interactions  and  overell  safety 
over  performance  factors  which  are  associated 
with  operational  fleet  aircraft.  These  were  im¬ 
portant  considerations  in  refining  the  baseline 
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Ratings  for  Academics  -  In  reviewing  the  3M5 
ratings  of  academic  characteristics,  we  observe 
some  spread  between  the  leader,  "media  update 
capability",  and  the  least  important  characteristic, 
"multiple  instructional  strategies".  However, 
reviewing  the  bar  chart  in  Pig,  4,  we  see  that  the 
differences  are  not  as  striking  as  in  the  aircraft 
and  simulator  areas.  In  any  event,  from  ar.  over¬ 
all  system  effectiveness  viewpoint,  the  ratings 
are  valuable  for  reaching  conclusions  regarding 
design  options. 

Criticality  Factors 

The  concept  of  criticality  as  used  in  the 
effectiveness  methodology  addresses  the  uniqueness 
of  each  learning  stage.  The  question  asked  is; 

"How  important  is  each  learning  stage  or  course 
relative  to  the  others"?  If  the  raw  SME  ratings 
were  used  in  the  analysis,  we  would  be  assuming 
that  all  learning  atages  had  a  criticality  factor 
of  one;  that  is,  they  were  all  of  equal  importance. 
However,  there  is  a  history  of  concern  among  train¬ 
ing  people  about  this  point.  Indeed,  a  report  was 
published  by  CNATRA  dealing  with  a  related  study 
(4).  This  report  describes  the  use  of  a  task  in¬ 
ventory  questionnaire  that  was  utilized  as  a  data 
collection  vehicle.  Evaluators  rated  training 
tasks  as  to  their  frequency,  criticality  and  ade¬ 
quacy.  The  results  are  interesting,  but  not  di¬ 
rectly  applicable  to  the  new  system  under  con¬ 
sideration  here.  We  are  faced  with  the  problem  of 
developing  a  weighting  scheme  for  measuring  criti¬ 
cality  in  the  VTXTS  program. 

Time  Value  Ratio  -  The  Time  Value  Ratio  at¬ 
tempts  to  determine  learning  stage  criticality  by 
relating  it  to  time  consumed  per  learning  stage. 

At  the  beginning  of  the  study  this  technique  was 
planned  for  use  on  the  TIO  basis.  When  the  matrix 
was  modified  to  incorporate  learning  stages  rather 
than  TIjOs,  the  Time  Value  Ratio  was  modified  ac¬ 
cordingly,  The  relative  importance  of  the  learn¬ 
ing  stage  was  calculated  by  taking  the  ratio  of 
time  devoted  to  it  divided  by  the  total  training 
time.  For  example,  if  a  partic'.ilar  learning 
stage  consumes  70  flight  training  hours  and  the 
total  flight  training  consists  of  170  hours,  then 
the  Time  Value  Ratio  would  be  0.4l.  For  the  ini¬ 
tial  runs  of  the  Effectiveness  Model,  criticality 
factor  was  defined  as  Time-Value  Ratio.  Modi¬ 
fications  were  to  be  incorporated  during  the  study, 
as  time  alone  is  not  the  only  factor  in  determining 
the  importance  of  a  learning  stage.  This  was  at¬ 
tempted  with  a  questionnaire  that  introduced  such 
factors  as  task  performance  difficulty,  task  famil¬ 
iarity,  extent  to  which  task  must  ue  completed  be¬ 
fore  progressing  in  the  training  process,  and 
eventual  mission  impact.  The  questionnaire  was 
not  exhaustive ;  to  include  all  the  factors  would 
be  a  prohibitive  task  considering  the  scope  of 
the  study.  The  pairwise  comparison  discussed  in. 
the  following  subsection  provided  an  approach 
which  directly  evaluated  learning  stage  criti¬ 
cality. 

Pairwise  Comparison  for  Criticality 

The  general  approach  is  to  make  a  comparison 
by  pairs  of  all  of  the  learning  stages,  and  judge 
which  is  the  more  important  (i.e,,  critical)  of 
each  pair.  After  all  the  possible  pairs  have  been 
compared,  with  no  ties  permitted,  the  number  of 
"wins"  is  counted  for  each  learning  stage.  Using 


a  matrix  technique,  the  weighting  factors  are  cal¬ 
culated  taking  into  account  the  relative  strength 
of  the  individual  "wins". 

The  two  basic  parts  of  the  weighting  factor 
(criticality)  calculation  are  the  pairwise  com¬ 
parison  of  the  learning  stages,  and  the  formu¬ 
lation  and  manipulation  of  a  dominance  matrix. 

Since  the  comparisons  are  made  on  a  pairwise  basis, 
the  total  number  of  "matches"  is  the  combination 
of  the  number  of  learning  stages  taken  two  at  a 
time.  A  list  of  all  the  pairings  is  prepared, 
then  the  rater  indicates  which  one  of  each  pair 
is  the  mare  important.  The  degree  of  importance 
is  not  asked.  The  relative  importance  is  taken 
care  of  by  the  dominance  matrix  manipulations . 
Ideally,  the  rater  should  be  able  to  support  his 
choices.  A  consensus  of  several  raters  is  desired 
In  this  case  all  of  the  raters  were  our  consultant 
pilots.  Differences  were  resolved  by  round-table 
discussions  with  each  rater  defending  his  choice. 
Assuming  the  choices  are  properly  supported,  a 
degree  of  objectivity  is  realized. 

Perhaps  the  easiest  way  to  explain  the 
philosophy  of  the  weighting  calculations  is  to 
draw  a  parallel  to  a  ’'round  robin"  tournament. 

In  the  scoring  system,  we  wish  to  take  into 
account  not  only  the  number  of  wins  a  team 
has,  but  the  strength  of  the  teams  beaten. 

In  general  teiros,  we  credit  a  team  with  its 
own  wins,  and  the  number  of  wins  of  eaoh 
beaten  team.  Assume  that  teams  A  end  B  have 
one  win  each,  over  C  and  D.  The  we  differen¬ 
tiate  between  A  and  B  by  counting  the  C  and 
D  wins  and  adding  these  to  the  A  and  B  wins. 

The  exact,  procedure  with  required  derivations 
is  described  in  Reference  3.  The  resulting 
criticality  factors  are  shown  in  Figure  5. 

The  product  of  the  importance  rating  and 
the  criticality  factor  is  defined  as  utility 
rating.  Utility  rating  for  aircraft,  simu¬ 
lators,  and  academics  were  calculated  and  pre¬ 
sented  in  our  study  report  in  a  raatrlce  format. 

The  values  within  the  matrices  are  a  measure 
of  the  importance  of  characteristic  in  accomplish¬ 
ing  a  learning  stage  combined  with  the  relative 
significance  of  the  learning  stage  in  the  overall 
training  curriculum.  The  larger  the  rating  value, 
the  greater  the  training  utility  associated  with 
the  rating. 

Bystem  Amplification  Factor 

To  evaluate  how  well  alternative  systems 
accomplish  the  training  goals,  some  measure  of 
their  value  must  be  introduced.  For  any  charac¬ 
teristic  of  the  system,  the  ratio  of  an  alterna¬ 
tive  value  to  the  baseline  characteristic  value 
is  defined  as  the  system  amplification  factor, 
or  value  ratio,  for  that  characteristic. 

Effectiveness  Computations 

The  Training  Effectiveness  Model  performs 
two  separate  and  sequential  computations :  the 
utility  ratings  of  the  system  characteristics 
and  the  training  effectiveness  ratios  of  al¬ 
ternative  systems.  For  the  VTXTS,  the  utility 
rating  computation  is  performed  once  and  the 
results  remain  fixed.  However,  the  effectiveness 
computation  is  performed  repetitively  in  develop- 
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Learning  Stage  Criticality  Factors 
(Pairwise  Comparison) 

Weighting  factors  applied  to  learning  stages. 

Learning  Stagat 

Aircraft  &  Simulators 

Familiarization  -  Stage  1 

0.191 

Familiarization  -  Stag*  2 

0.024 

Basic  Instrumentation 

0.113 

Radio  Initrumantation 

0.073 

Airways  Navigation 

0.042 

Formation 

0.109 

Gunnary 

0.026 

Night  Familiarization 

0.029 

Oparational  Navigation 

0.044 

Weapons 

0.088 

Air  Combat  Manauvar 

0.086 

Carriar  Qual  1 

0.148 

Carrier  Qual  II 

0.028 

Learning  Stages 

Academics 

Flight  Support 

0.232 

Aviation  Student  Info 

0.014 

Engineering 

0.170 

Aerodynamics 

0.109 

Meteorology 

0.109 

Flight  Rules  &  Reg 

0.148 

IFR  NAV 

0.126 

Oparational  NAV 

0.092 

Figure  5 


ing  and  refining  the  baseline  system  FOM  and  in 
comparing  alternative  systems  to  the  baseline. 
The  output  format  of  the  effectiveness  computa¬ 
tions  is  shown  in  Fig.  6. 

In  the  figure  ALT  VAL  and  BAS  VAL  are  the 
characteristic  values  for  the  alternative  and 
baseline  systems  respectively.  BAS  FOM  i3  the 
training  utility  of  each  baseline  characteristic 
as  obtained  from  the  utility  rating  computation 

The  amplification  factor  or  value  ratio, 

VAL  RTO,  is  the  ratio  of  ALT  VAL  to  BAS  VAL  of 
each  characteristic.  The  product  of  BAS  FOM  and 
VAL  RTO  is  the  corresponding  training  utility, 
ALT  FOM,  for  each  characteristic  of  the  alter¬ 
native  system.  The  effectiveness  ratio  of  an 
alternative  system  element  is  the  average  alter¬ 
native  system.  The  effectiveness  ratio  of  an 
alternative  system  element  is  the  average 
alternative  FOM  divided  by  the  average  baseline 
FOM. 

Finally,  the  alternative  system  effective¬ 
ness  ratio  is  computed  and  displayed  as  the  sum 
of  the  average  FOMs  of  the  alternative  system 
elements  divided  by  the  sum  of  the  average  FOMs 
of  the  baseline  system  elements. 

System  Comparisons 


potential  of  two  systems,  each  consisting  of  a 
training  aircraft,  a  simulator  suite  and  an 
academics  program,  once  the  characteristic 
utility  ratings  are  established,  it  remains  to 
evaluate  the  characteristics  of  both  systems, 
one  of  which  is  taken  to  be  the  reference  or 
baseline  system. 

Figure  6  illustrates  such  a  comparison.  The 
baseline  system  is  the  Grumman/Boech  ='"-cem  730 
as  envisaged  at  the  start  of  the  .  ^  ASE  study. 

The  alternative  system  is  the  refined  Grumman/ 
Beech/Link  System  730  as  it  emerged  at  the  comple¬ 
tion  of  the  study.  An  increase  in  training 
effectiveness  ratio  has  been  achieved  in  ail 
three  system  elements.  Individual  element  improve¬ 
ments  amount  to  1^,7%  in  the  aircraft,  28.8%  in 
the  simulators  and  *+3.1%  in  academics.  Overall, 
the  refined  System  730  shows  a  29.9%  increase  in 
training  effectiveness  relative  to  the  baseline 
system. 

A  similar  comparison  was  made  between  the 
baseline  System  730  and  the  Naval  Integrated 
Flight  Training  System  (NIFTS).  NIFTS  is  the 
current  undergraduate  Flight  Training  System. 

Since  NIFTS  utilizes  two  aircraft,  the  T-2C  for 
intermediate  training  and  the  TA-Uj  for  advanced 
training,  characteristic  values  for  both  aircraft 
were  used.  Therefore,  the  computation  of  training 
effectiveness  ratio  for  NIFTS  was  based  upon  the 
proportional  contribution  of  each  aircraft  to  the 
individual  learning  stages  in  the  program. 

The  results  of  this  comparison  are  summarized 
in  Figure  7.  The  results  show  that  the  baseline 
System  730  envisaged  at  the  start  of  the  ASE  study 
provided  a  31.9%  improvement  in  training  effective¬ 
ness  over  NIFTS  and  the  refined  System  730  offers 
a  71. *1%  improvement  over  NIFTS.  Improvements  in 
the  individual  elements  are  indicated  also. 


Sensitivity  Analysis 

A  salient  feature  of  the  Training  Effective¬ 
ness  Model  is  its  ability  to  compare  conceptual 
systems.  If  the  characteristic  measurement  values 
are  not  established,  parametric  value  ratios  are 
used  to  determine  training  effectiveness  sensi¬ 
tivity  to  variations  of  characteristic  values. 
Figure  8  shows  the  training  effectiveness  sensi¬ 
tivity  of  the  aircraft,  simulators  and  academics 
to  value  ratio  variations.  The  relationships  are 
established  by  varying  the  value  ratio  of  one 
characteristic  at  a  time  and  recording  the  per¬ 
cent  change  in  the  system  effectiveness  ratio. 

This  type  of  senritivity  analysis  identifies  the 
element  characteristics  offering  the  greatest 
potential  for  improvement  in  effectiveness  ratio. 

The  Training  System  Effectiveness  Model,  is 
being  used  as  a  management  tool  to  evaluate  the 
relative  merit  of  competing  systems.  The  major 
comparison  considered  the  growth  in  effectiveness 
of  the  Grumman/Beech/Link  System  730  as  it  pro¬ 
gressed  from  the  baseline  to  the  refined  design. 
The  characteristic  changes  accounted  for  a  growth 
of  29.9%  in  effectiveness  by  the  conclusion  of  the 
ASE  study.  The  refined  System  730  is  shown  to  be 
7-1  !r%  more  effective  than  NIFTS. 


The  Training  Effectiveness  Model  has  the 
capability  of  comparing  the  relative  training 
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System  Comparisons 

Compares  training  effectiveness  of  baseline  and 
refined  System  730  with  NIFTS. 
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WRITING  AN  ISO  TRAINING  PROGRAM  CONCURRENTLY 
WITH  FULL  SCALE  DEVELOPMENT 

CAPT  JOSEPH  C.  STEIN 
CAPT  WALTER  P.  PETROFSKI 
OAS/CRUISE  MISSILE  TRAINING  MANAGERS 
CARSWELL  AFB,  TX  76127 

ABSTRACT 

Developing  a  training  program  concurrently  with  full-scale 
development  of  a  new  system  has  been  looked  upon  by  several 
educational  professionals  as  being  Impossible.  The  US  Air  Force 
desire  to  shorten  the  time  from  drawing  board  to  full  operational 
capability  for  a  weapon  system  requires  combining  normal  system  life 
phases  whenever  possible.  Therefore,  the  4235th  Strategic  Training 
Squadron  was  tasked  to  develop  a  training  program  for  the  Offensive 
Avionics  System  (OAS )  and  Air  Launch  Cruise  Missile  (ALCM) 
Modifications  to  the  B-52  G  and  H  fleets  concurrently  with  full-scale 
development  of  hardware  and  software  for  the  aircraft.  Having  written 
the  training  program  using  the  ISD  approach,  the  squadron  provides 
Insight  into  the  problems  of  such  an  effort  and  the  solutions  It 
developed  to  overcome  those  problems.  Areas  addressed  In  this  paper 
include: 

(1)  Developing  a  core  of  knowledge  about  a  new  system  without 
an  established  working  model. 

(2)  Selecting  an  organizational  method  for  presenting  the 
training  program. 

(3)  Developing  training  devices. 

(4)  Developing  technical  orders. 

(5)  Handling  of  changes  to  system  operation  and  implementing 
them  Into  the  training  program. 

(6)  Selecting  Instructional  media. 

(7)  Selecting  Instructors. 


INTRODUCTION 


Since  1977  the  U.S.  Air  Force  has  prepared  to 
integrate  the  Offensive  Avionics  System  (OAS)  and 
Air  Launched  Cruise  Missile  (ALCM)  Into  the  B-52 
G  and  H  model  aircraft.  With  the  date  for 
delivery  of  the  first  operational  aircraft  set 
for  August  1981,  preparation  for  its  arrival  had 
to  be  expedited  drastically.  To  do  so  several 
phases  of  normal  system  development  had  to  be 
accomplished  simultaneously. 

Along  with  full-scale  development  and  flight 
testing  the  prototype  system,  a  training  program 
had  to  be  written  for  the  aircrew  member. 
Despite  the  fact  that  many  educators  believe  such 
a  simultaneous  effort  Is  Impossible  It  had  to  be 
done.  The  4235th  Strategic  Training  Squadron 
(STS)  actually  began  working  on  the  training 
program  using  the  ISD  approach  In  April  1978. 
The  training  program  developers  have  experienced 
numerous  problems  throughout  their  three  years  of 
work.  However,  they  have  overcome  them  and  are 
ready  to  Implement  their  program  to  the 
crewmembers  for  the  conversion  to  the  new 
equipment.  The  squadron  can  now  provide  Insight 
into  the  problems  they  encountered,  the  solutions 
they  used,  and  additional  suggestions  to  help  In 
similar  circumstances. 


DEVELOPING  A  CORE  OF  KNOWLEDGE 

One  of  the  most  obvious  problems  that  arises 
when  trying  to  create  a  training  program 
simultaneously  with  full-scale  development  of  the 
equipment  Is  building  a  core  of  knowledge  about 
the  system.  Early  In  1980,  almost  8  months 
before  the  first  OAS/ALCM  equipped  B-52  flight, 
seven  Instructor  radar  navigators  were  assigned 
to  the  4235th  Strategic  Training  Squadron  at 
Carswell  AFB,  Texas  to  begin  preparing  the 
training  program. 

First,  all  the  training  program  developers 
completed  an  Instructional  Systems  Development 
course  given  by  the  squadron.  This  course 
provided  the  basic  understanding  necessary  for 
all  the  writers  to  work  toward  the  same  goals 
using  the  same  methods.  This  proved  to  be  an 
extremely  essential  ingredient  In  developing 
continuity  and  uniformity  In  the  various  training 
blocks. 

Next,  the  training  program  developers  began 
reading  the  numerous  engineering  documents 
published  by  the  contractors.  These  documents 
provided  In-depth  explanations  of  all  the 
OAS/ALCM  operations.  However,  it  must  be  made 
clear  that  it  was  not  necessary  to  have  an 
engineering  degree  to  understand  these  documents. 
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The  1SD  course  proved  valuable  during  this  phase 
also  because  each  training  developer  was  now  able 
to  evaluate  each  bit  of  knowledge  as  to  Its  later 
usefulness  to  the  student.  After  six  weeks  of 
studying  documents  the  next  phase  began. 

A  task  analysis  of  each  activity  the 
navigators  of  a  B-52  would  perform  was  compiled. 
This  Included  both  the  tasks  associated  with  the 
new  equipment  and  those  In  which  the  new 
equipment  did  not  come  Into  play.  This  phase  of 
preparation  served  two  purposes.  First,  It 
Identified  the  Items  that  had  to  be  taught  to  the 
navigators  so  they  could  convert  to  the  new 
equipment.  Second,  It  translated  the  operation 
of  the  OAS  from  engineering  vernacular  to 
understandable  English.  This  phase  took 
approximately  four  months  to  complete. 

The  final  source  of  Information  prior  to  the 
first  simulator  or  aircraft  having  OAS  equipment 
was  the  familiarization  course  and  operator's 
course  conducted  by  Boeing  (Type  I  Training). 
Although  the  course  that  was  taught  encountered 
the  same  problems  of  not  having  a  simulator  or 
aircraft  to  verify  the  system's  actual  operation. 
It  did  provide  some  additional  Information.  It 
also  allowed  the  training  personnel  to  meet  the 
engineers  who  wrote  the  software.  This  was 
Important  because  when  questions  arose  later  the 
training  developers  knew  wi.o  to  call  for  a 
possible  answer. 

When  the  aircraft  finally  made  Its  maiden 
flight  training  personnel  had  already  begun  the 
actual  development  of  their  program.  As  the 
aircraft  continued  Its  test  program,  dally  phone 
conversations  with  the  flight  personnel  provided 
constant  updates  to  the  training  program. 
Ironically,  the  Information  flow  was  also 
reversed  at  times.  The  training  developers 
having  spent  so  much  time  preparing  to  build 
their  program  were  able  to  provide  flight  test 
personnel  with  Information  how  the  engineering 
documents  said  the  OAS  should  operate.  This 
allowed  flight  test  to  Identify  numerous  actual 
system  operation  errors  which  needed  correction. 

Combining  their  early  efforts  In  learning  the 
engineering  documents  and  the  updates  from  flight 
test,  the  training  program  progressed  on 
schedule.  Having  almost  completed  the  training 
program  several  recommendations  can  be  drawn  from 
the  4235th  STS  experience: 

(1)  All  training  program  developers 
should  use  the  same  basic  instructional  approach 
within  a  program. 

(2)  The  training  program  should  start  as 
early  as  possible  so  personnel  can  develop  a 
sound  knowledge  of  the  system  to  be  taught. 

(3)  Training  program  developers  should 
attend  all  available  contractor  training  but  only 
after  they  know  a  good  deal  about  the  system 
operation.  This  provides  a  framework  for 
understanding  the  system  rather  than  mere 
acceptance  of  the  contractor's  claims. 

(4)  In  the  case  of  simultaneous 
accomplishment  of  several  phases  of  system 


development  It  Is  essential  that  the  test 
personnel  for  the  system  be  highly  qualified  and 
knowledgeable  In  the  expected  system  operation 
prior  to  the  actual  test  phase.  Since  the 
engineering  documents  are  the  only  basis  for 
training  programs  development,  test  personnel 
must  be  Intimately  familiar  with  them.  Then  they 
can  not  only  Identify  system  errors  and 
shortcomings  but  also  provide  timely  Information 
to  training  personnel  about  discrepancies  from 
the  documents.  Early  Identification  of  such 
dlrcrepancles  can  be  handled  by  changes  to  the 
training  program  or  by  placing  the  demand  on  the 
contractors  to  correct  the  problem.  If  a  strict 
deadline  Is  set  for  bringing  the  system  on  line. 
It  becomes  more  difficult  to  make  such  changes  to 
the  equipment.  Therefore  the  equipment  Is 
delivered  with  less  than  the  original 
capabilities  and/or  the  operator  must  use 
workaround  procedure* -to  achieve  his  objective. 

(5)  When  descrepancles  are  Identified  by  the 
test  or  training  personnel  the  system  manager 
must  support  the  effort  to  rewulre  the  correction 
of  the  deficiency.  If  It  Is  not  required  that 
all  deficiencies  be  corrected,  a  weekly  report  of 
such  deficiencies  must  be  published.  This 

provides  all  other  agencies  with  data  to  modify 
their  efforts. 

SELECTING  AN  ORGANIZATIONAL  METHOD 

After  developing  a  core  of  knowledge  about 
how  the  OAS  and  ALCM  operated  the  training 

program  developers  had  to  choose  an 
organizational  pattern  for  the  training  program. 
The  decision  came  down  to  two  alternatives  -  the 
systems  approach  and  the  phase-of -flight 

approach.  Traditionally,  the  Air  Force  has  used 
the  systems  approach  for  most  aircrew  training 
programs.  Using  this  approach  the  training 

program  breaks  the  large  system  Into  subsystems 
(radar,  doppler,  heading,  etc)  and  then  teaches 
the  student  all  he  needs  to  know  about  the 
subsystem. 

After  all  the  subsystems  are  explained  the 
student  Is  expected  to  have  an  understanding  of 
the  overall  system  operation.  However,  more 
often  than  not  this  approach  leaves  the  student 
with  fragmented  system  knowledge  and  requires  a 
lengthy  period  of  time  for  complete  knowledge  of 
the  system  to  be  obtained.  The  other 
alternatives  phase-of-fllght  provided  a  much  more 
task-oriented  approach  for  the  training  program. 
Beginning  with  mission-planning  the  student  would 
proceed  through  preflight,  enroute  procedures, 
descent,  landing,  and  post-flight.  This  approach 
allowed  the  program  developers  to  concentrate  on 
developing  procedures  and  checklists  as  well  as 
transmitting  knowledge  to  the  student. 
Therefore,  the  student  left  each  segment  of 
training  knowing  both  the  knowledge  he  needed  and 
how  to  use  It.  Because  the  conversion  of  each 
crew  member  to  the  OAS/ALCM  was  to  be  done  In 
only  three  to  four  flights,  the  task-oriented 
phase-of-fllght  approach  was  selected  for  the 
training  program  organizational  pattern. 

As  was  stated  earlier  the  initial  step  In 
actually  writing  the  training  program  was 
completing  a  task  analysis  for  each  activity 
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FIGURE  1 


accomplished  by  the  crew  members.  Formatted  and 
entered  In  a  word-processor  this  document  formed 
the  skeleton  for  the  body  of  the  training 
program.  Figure  1  shows  a  task  analysis  for  one 
of  the  tasks  during  pref light.  The  key  explains 
how  this  document  was  used  for  the  conversion 
training  and  how  It  will  be  used  for  the  Combat 
Crew  Training  School  curriculum  which  must  be 
developed.  The  phase-of-f light  approach  was 
especially  well-suited  for  organizing  this  task 
analysis  which  now  could  be  used  as  an  outline  of 
the  training  program. 

Regardless  of  which  pattern  was  chosen  the 
simultaneous  accomplishment  of  full-scale 
development  and  writing  a  training  program  would 
have  caused  problems.  The  systems  approach  would 
naturally  be  more  suited  to  handling  the  constant 
changes  of  the  OAS  software.  A  training  program 
developer  could  simply  find  the  training  block 
devoted  to  the  changed  equipment  and  make  the 
required  changes.  The  phase-of-fl Ight  approach 
on  the  other  hand  required  locating  all  the  tasks 
involved  with  a  particular  change  and  making  the 
corrections.  Although  the  task  analysis  format 
used  In  this  program  aided  In  the  change  process, 
corrections  did  take  more  time  to  insure  all 
tasks  were  correct.  However,  once  made,  the 
change  helped  stimulate  reexamination  of 
checklists  and  procedures.  These  crosschecks 
aided  In  delivery  of  a  training  package  that 
agreed  with  checklists  and  the  operator  manuals. 

The  problems  of  handling  the  constant  changes 
to  OAS  software  required  flexibility  In 
curriculum  development.  As  each  change  notice 
arrived  the  appropriate  task  analysis  and 
training  lesson  was  updated.  However,  It 
surfaced  that  just  teaching  the  basic  tasks  to 
the  crew  members  failed  to  give  him  the  "big 
picture"  of  managing  the  navigational  computers. 
To  provide  this  insight  required  a  departure  from 
the  task-oriented  format.  After  the  functional 
descriptions  of  the  switches  and  equipment  used 
during  preflight  and  before  takeoff,  the  student 
needed  an  understanding  of  what  he  was  trying  to 
achieve  in  programming  the  navigational  computers 
and  the  best  way  to  go  about  doing  so. 
Therefore,  a  lesson  was  written  which  explained 
the  techniques  and  theory  of  Inertial 
navigational  equipment.  However,  this  theory  was 
extremely  limited  in  scope  and  operator  oriented. 
Collecting  the  data  for  such  a  block  proved 
difficult  because  empirical  data  did  not  exist 
due  to  the  limited  experience  of  flight  test. 
Instead,  the  procedures  for  basic  management  of 
the  system  had  to  be  derived  from  software 
documents  and  telephone  conversations  with  Boeing 
programmers  and  engineers.  The  important  aspect 
of  this  effort  was  not  the  magnitude  of  the  work 
Involved  but  the  realization  that  training 
personnel  must  be  flexible.  The  objective  should 
be  to  make  the  student  as  capable  of  performing 
the  desired  task  as  possible.  With  limited  time 
to  achieve  this  objective  willingness  to  depart 
from  standard  procedures  must  be  present.  If  a 
systems  approach  is  more  suitable  for  a  training 
block  then  It  must  be  used.  If  theory  must  be 
taught  In  a  task  oriented  program  -  do  It.  The 
student  performing  the  task  Is  the  ultimate.  To 
argue  over  educational  concepts  or  delay 
completion  of  a  training  program  to  spend 


excessive  time  to  maintain  purity  of  the  training 
approach  Is  Inexcusable.  In  the  OAS/ALCM 
training  program  this  flexibility  saved 
invaluable  time  In  meeting  the  deadline  set  for 
aircrew  training.  It  also  provided  a  period  of 
time  to  review  the  program  and  publish  last 
minute  errata. 

DEVELOPING  TRAINING  DFVICES 

As  the  training  requirements  for  the  OAS  and 
AI.CM  were  developed.  It  became  apparent  that 
lower  level  tasks  and  systems  functions  would 
have  to  be  trained  in  a  ground  simulator.  The 
first  question  that  arose  pertained  to  what 
simulator  could  be  used.  The  WST  designed  for 
the  B-52  would  not  be  ready  for  the  OAS 
conversion  and  the  present  T-10  could  not  be 
modified  due  to  current  training  requirements. 
Using  the  ISO  approach,  the  4235th  STS  developed 
a  list  of  tasks  which  required  training  In  this 
ground  simulator.  These  tasks  and  trainer 

requirements  were  determined  with  both  the  OAS 
conversion  program  and  the  Combat  Crew  Training 
School  program  In  mind.  The  tasks  were 
prioritized  from  simple  to  hard.  A  simple  task 
consisted  of  anything  from  knowledge  of  switch 
locations  to  a  3  -  5  step  system  operation 
proceudre.  The  hearder  tasks  consisted  of 
weapons  supervision  and  OAS  system  management. 
Investigation  of  these  tasks  determined  that  a 
series  of  training  devices  could  be  developed. 
For  the  OAS/ALCM  program  two  devices  were 
necessary.  The  first  device  would  be  used  to 
train  simple  tasks  or  lower  level  objectives. 
The  second  device  would  be  used  to  train  simple 
tasks  or  lower  level  objectives.  The  second 
device  would  be  quite  sophisticated  and  would 
train  higher  level  objectives.  The  two  devices 
would  compliment  each  other  and  the  weapon 
system. 

The  OAS  training  managers  decided  to  use  a 
Cockpit  Familiarization  Trainer  (CFT)  for  lower 
level  objectives  and  a  Part  Task  Trainer  (PTT) 
for  the  higher  level  objectives.  As  the 
requirements  and  designs  of  the  devices  began  to 
take  shape,  it  became  readily  apparent,  that 
these  devices  would  be  the  backbone  of  the  OAS 
Conversion  Training  Package. 

The  CFT  was  designed  as  a  wooden  mock-up  of 
the  B-52  OAS/ALCM  modified  Radar  Navigator  and 
Navigator  stations.  All  the  panels  in  the  CFT 
are  Inoperative  with  visible  pushbutton  switch 
legends.  Two  of  the  four  multifunctional 
displays  are  also  inoperative;  however,  two  are 
replaced  with  Singer  Caramate  II  Tape/Slide 
Projectors.  As  stated  before,  this  device  will 
be  used  to  teach  switch  locations  and  minor 
procedures. 

The  student  will  select  a  tape/slide  program 
and  view  It  on  the  projector  In  the  CFT.  As  the 
program  directs  him  to  a  certain  switch  or  series 
of  switches,  he  is  able  to  locate  them  In  a  more 
real  environment.  The  student  can  run  checklists 
and  even  take  step  by  step  actions  necessary  to 
perform  minor  OAS  procedures. 

The  design  of  this  device  Is  simple  but  the 
training  Impact  Is  great.  Another  Important 
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design  feature  In  the  CFT  Is  mobility.  This 
device  must  be  capable  of  being  moved  to  the 
various  sites  where  the  OAS/ALCM  training  program 
will  be  conducted. 

The  B-52  OAS/ALCM  Part  Task  Trainer  Is  a 
sophisticated  device  that  addresses  some  of 
tasks,  procedures  and  conditions  the  crew  members 
must  handle  with  the  actual  weapon  system  (see 
FIGURE  2).  Because  the  aircrews  receiving 
training  will  be  highly  experienced  with  B-52 
navigational  functions,  the  focus  of  the  PTT  will 
be  on  procedures  training;  specifically,  those 
procedures  unique  to  the  new  OAS.  As  an  example, 
the  PTT  must  respond  exactly  as  the  OAS  would  to 
all  commands  (button  pushes,  switch  activations, 
etc.),  but  only  those  features  used  directly  In 
navigational  and  weapon  procedures  need  be 
displayed  In  radar  video. 

The  physical  layout  of  the,  the  PTT  provides 
stations  for  the  two  crewmembers  required  to 
operate  the  OAS,  as  well  as  for  an  Instructor. 
The  PTT  Is  a  physical  mockup  Identical  to  the  OAS 
layout.  The  mockup  Is  comprised  of  16 
operational  panels,  two  of  which  Include  a 
keyboard  and  trackball  (one  for  each  crewmember). 
Four  monochromatic  display  monitors  provide  up  to 
33  different  display  formats,  three  of  which 
Include  synthetic  radar  imagery.  The 

instructor's  position  is  equipped  with  a  CRT 
console  for  setup  and  monitoring  of  the  training 
sessions. 

The  only  problems  In  training  device 
development  occurred  when  changes  were  made  to 
the  OAS  hardware  (panels/aircraft  configuration) 
and  software  (program  and  system  operation). 
Since  the  CFT  was  being  constructed  locally,  any 
changes  that  were  received  were  Incorporated 
directly  to  the  design.  The  panels  being  static 
could  be  changed  or  rebuilt  with  relative  ease. 
Any  change  In  cockpit  configuration  could  be  done 
easily.  Software  changes  did  not  effect  the  CFT. 

The  PTT  with  Its  full -working  mockup  was 
effected  greatly  by  both  software  and  hardware 
changes.  A  system  to  handle  these  changes  was 
established  between  the  4235th  STS  and  the  other 
agencies  Involved  In  the  project.  As  in  the  CFT, 
the  PTT  crewstation  was  constructed  locally  so 
changes  could  be  Implemented  with  minor 
coordination  efforts.  The  software  changes  had  a 
much  greater  Impact  on  the  PTT  design.  A  change 
in  the  OAS  operational  system  put  the  PTT 

software  one  step  behind  the  aircraft. 

To  handle  this  problem  the  simulator  manager 
set  up  coordinated  communication  between  all 
agencies  to  determine  how  the  changes  impacted 

the  software  design.  The  4235th  STS  received 
changes  from  Boeing  and  determined  what  impact 
they  had  on  the  PTT  and  the  training  program.  If 
the  change  was  significant,  a  written  change 
request  was  forwarded  to  the  other  agencies 

Involved.  The  software  specialist  would  act  on 
the  request  and  forward  a  yes/no  proposal  for 
Impl (mentation.  For  the  prototype  software  a 
freeze  date  finally  had  to  be  established.  All 
software  changes  were  accepted  until  this  date. 
After  this  date,  all  changes  were  held  at  the 
4235th  STS  for  further  evaluation.  After 


completion  of  the  prototype  software  effort,  any 
remaining  changes  would  be  Implemented  In  the 
baseline  software  effort  for  the  first  production 
device.  By  developing  a  prototype  and  a  baseline 
software  a  minimum  number  of  crews  would  be 
effected  by  differences  between  the  PTT  and  the 
aircraft. 

The  software  design  of  the  PTT  was  based  on 
the  modular  design  concept  which  made  software 
changes  easy.  This  gave  the  PTT  the  flexibility 
necessary  to  Incorporate  changes. 

DEVELOPING  TECHNICAL  ORDERS 

One  of  the  associated  Items  Involved  with 
bringing  any  system  Into  operation  is  the 
development  of  manuals  (tech  orders)  for  the 
operator.  Recognizing  that  manuals  written  for 
other  aircraft  systems  often  became  bogged  down 
In  engineering  jargon  and  a  conglomeration  of 
useless  facts.  Air  Force  personnel  Involved  in 
the  early  stages  of  the  OAS/ALCM  Integration 
Insured  training  personnel  would  play  an 
important  role  In  the  new  manuals.  Even  before  a 
working  model  of  the  OAS  was  available  or  the 
ALCM  had  flown,  meetings  were  held  to  organize 
the  development  of  the  aircrew  manuals. 

These  meetings  were  essential  for  several 
reasons.  They  established  that  the  books  were  to 
be  written  for  the  OAS/ALCM  operator  not  the 
staff  planners  or  weapons  officers.  Therefore, 
extraneous  material  was  deleted  or  placed  In 
other  manuals  used  by  those  Individuals. 
Readability  and  functionality  of  the  manuals  was 
also  stressed  to  Boeing  writers.  This  ensured 
the  engineering  language  would  not  be  present 
since  most  crewmembers  lacked  such  backgrounds. 
Also  Important  was  the  fact  that  the  books  would 
be  used  by  the  fully-qualified  crewmembers. 
Therefore,  the  books  did  not  have  to  take  a  tone 
of  training  documents.  This  eliminated  the  need 
for  repeated  explanations  of  basic  operations 
such  as  keyboard  entries.  Through  the  meetings 
these  cosmetic  changes  became  easier  to  make  as 
Boeing  writers  learned  the  desires  of  the  Air 
Force.  Each  published  change  underwent  editorial 
and  technical  reviews  by  Air  Force  and  Boeing 
personnel.  As  each  change  was  published  these 
reviews  required  less  time  due  to  the  parties 
understanding  each  others  problems  and  desires. 

However,  the  largest  pay  off  from  these 
meetings  came  from  the  communications  established 
between  the  Boeing  writers,  flight  test 
personnel ,  and  the  4235th  STS.  Sometimes 
bordering  on  harassement,  the  Interplay  caused 
several  hundred  questions  to  arise  about  the 
system's  operation.  Procedures  and  checklists 
were  developed  which  would  have  otherwise  taken 
months  to  evolve  without  the  collective  knowledge 
of  the  grouos.  However,  despite  the  success 
brought  by  early  efforts  there  is  no  denying  that 
simultaneous  accomplishment  of  several  phases  of 
system  development  created  problems. 

Software  implementation  of  the  OAS/ALCM  did 
not  proceed  as  easily  as  anticipated.  Daily 
changes  to  the  software  programs  became 
commonplace.  This  caused  changes  to  the 
engineering  documents.  The  lag  between  actual 
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change  of  operation  of  the  OAS  and  changes  to  the 
engineering  documents  required  tech  order 
development  to  be  extremely  flexible.  To  keep  up 
with  these  changes  not  only  In  the  training 
program  but  also  In  the  manuals  required  almost 
dally  telephone  communication  between  the  three 
groups.  Without  tnls  constant  conversing,  an  OAS 
would  have  been  delivered  that  bore  little 
resemblance  to  the  engineering  document,  the 
training  program,  or  the  tech  orders. 

In  addition  to  the  meetings  to  establish  the 
early  groundwork  for  communications,  several 
other  actions  could  ease  the  task  of  developing 
operator  tech  orders.  First,  tech  order  writers, 
test/test  flight,  and  training  personnel  should 
all  become  extremely  knowledgeable  concerning  the 
Information  In  the  engineering  documents.  Then 
they  all  should  attend  contractor  training  for 
the  new  system  (Type  I  Training  In  Air  Force 
terminology).  This  allows  them  the  opportunity 
to  ask  questions  concerning  unclear  areas  and  to 
be  questioning  students  rather  than  naive 
listeners  during  the  training.  Once  having 
completed  the  contractor  training  all  Individuals 
Involved  In  testing,  training,  and  publication 
writing  are  on  equal  ground  and  can  be  valuable 
Inputs  at  subsequent  meetings. 

Finally,  tech  order  wlters,  training  program 
developers,  test/test  flight  personnel,  and  a 
system  engineer  should  meet  as  often  as  possible 
to  discuss  how  the  system  Is  actually  working. 
In  the  case  of  an  aircraft,  this  should  be 
required  after  each  flight.  This  provides  the 
forum  for  discussion  of  system  operation  of  the 
engineering  documents.  It  also  provides  feedback 
to  all  parties  from  the  system  engineer 
concerning  any  changes  made  or  to  be  made  to 
those  documents,  on  a  more  timely  basis  than 
normal  printed  distribution  which  normally  takes 
weeks. 

HANDLING  SYSTEM  CHANGES 

Ideally  training  programs  written  after  a 
system  Is  developed  do  not  encounter  serious 
problems  with  changes  In  system  hardware  or 
software.  This  luxury  does  not  exist  In  a 
training  program  developed  simultaneously  with 
the  full-scale  development  of  the  system.  The 
daily  changes  In  the  Offensive  Avionics  System 
had  numerous  effects  on  the  training  program. 

Anticipating  a  system  resembling  the 
engineering  documents,  personnel  of  the  4235th 
STS  had  written  a  task  analysis,  lesson  outlines, 
and  simulator  mission  profiles  based  on  those 
documents.  Each  change  called  for  adjustments  to 
all  these  efforts.  The  changes  Initially  caused 
consternation  and  frustration  In  the  training 
program  developers.  However,  when  It  became 
apparent  that  changes  would  be  part  of  the  dally 
routine,  the  developers  learned  to  adapt  more 
quickly. 

A  few  suggestions  to  minimize  the  problems 
associated  with  the  handling  of  changes  to  the 
system  follow: 

(I)  Prepare  al_l_  Individuals  Involved 
with  the  training  program  for  the  changes  that 


will  more  than  likely  evolve.  This  means  not 
only  the  actual  writers  for  the  training  program 
but  also  the  Instructors  who  will  teach  It;  the 
graphics  people  who  must  prepare  new  slides  for 
what  seems  to  be  a  miniscule  change;  the 
Individuals  who  are  building  trainers,  etc.  This 
preparation  lets  everyone  know  that  the  changes 
are  not  generated  by  the  training  developers' 
errors  but  by  actual  changes  in  the  system.  This 
understanding  not  only  leads  to  Increased 
credibility  for  the  training  developers  but  also 
unites  everyone  Into  the  challenge  of  preparing  a 
quality  program  within  the  time  constraint 
Imposed. 

(2)  Establish  communication  lines 
between  all  parties  Involved  In  the  program.  The 
system  manager,  system  test  personnel,  training 
personnel,  system  englners,  manual  writers  and 
any  other  groups  concerned  with  system  operation 
must  meet  often  to  resolve  how  the  tystem  is 
operating  and  how  It  will  operate  once  In  the 
field.  Communications  lines  must  also  be 
established  to  allow  more  rapid  reaction  to 
problems  If  It  Is  required  between  such  meetings. 
In  Its  program  the  OAS  training  developers 
conducted  numerous  telephone  conferences  with  any 
agency  that  could  provide  up-to-date  Information 
concerning  OAS/ALCM. 

(3)  The  use  of  a  word  processor  was 
essential  In  keeping  the  training  lessons 
up-to-date.  When  a  writer  changed  a  lesson  It 
was  Immediately  put  Into  the  word  processor. 
This  provided  a  current  printout  of  any  lesson 
when  an  Instructor  asked  for  It. 

(4)  Realizing  that  It  Is  often  easier 
and  cheaper  to  change  an  engineering  document 
rather  than  hardware  and/or  software,  curriculum 
developers  must  develop  the  flexibility  to  change 
any  lesson.  However,  at  some  point  a  "freeze" 
must  be  Implemented  on  courseware  changes.  At 
this  point  coursebooks  or  multi -media  programs 
should  be  completed  and  all  changes  snould  take  a 
form  of  errata.  This  prevents  "the  last  minute" 
publication  of  courseware.  If  such  last  minute 
efforts  are  allowed  editorial  and  reproduction 
errors  will  create  the  appearance  of  lack  of 
professionalism.  As  was  said  euHler  -  changes 
due  to  system  development  can  be  explained. 
Sloppiness  In  controllable  areas  can  not  be 
excused. 

SELECTING  INSTRUCTIONAL  MEDIA 

Using  the  ISO  concept,  the  curriculum 
development  manager  usually  has  a  number  of 
Instructional  media  choices  to  deliver  his 
training  course.  In  selecting  the  proper 

instructional  media,  the  curriculum  manager  must 
select  the  media  that  is  the  best  to  convey  his 
subject  matter  to  the  student. 

In  developing  the  OAS/ALCM  Conversion 
Training  Program,  the  selection  of  Instructional 
media  was  limited  by  certain  constraints.  Many 
media  options  were  not  available,  became  to 
complex,  or  were  not  flexible  enough  to  handle 
program  changes.  Because  the  curriculum  was 
being  written  and  produced  during  system 
full-scale  development,  the  Instructional  media 
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had  to  be  simple  and  easy  to  update.  The 
training  manager  had  to  anticipate  system  changes 
and  be  able  to  update  the  courseware  quickly.  If 
the  media  chosen  for  a  lesson  was  to  complex,  It 
became  costly  In  both  time  and  dollars  to  change. 
The  curriculum  development  managers  of  the 
OAS/ALCM  program  used  simple  but  effective 
Instructional  media.  The  three  forms  of  media 
that  proved  flexible  enough  to  use  were 
tape/slldes,  coursebooks,  and  trainers  (CFT  or 
PTT ) .  All  three  of  these  proved  effective  and 
withstood  the  constant  change  of  a  developing 
system. 

SELECTING  INSTRUCTORS 

As  the  OAS  specifications  and  system 
requirements  were  completed  the  4235th  STS 
established  an  Initial  cadre  of  Instructors. 
These  seven  Instructors  were  to  function  as  both 
Subject  Matter  Experts  (SME)  and  Curriculum 
Development  Managers  (COM).  Once  assigned,  the 
Instructors  began  to  establish  a  core  of 
knowledge  by  studying  the  OAS/ALCM  specification 
and  Boeing  documentation.  Using  the  ISD 
approach,  the  seven  Instructors  were  tasked  to 
develop  the  entire  OAS/ALCM  Conversion  Training 
Program.  In  the  program  plan,  additional  people 
were  to  be  added  to  the  OAS  program  and  with  the 
original  seven  Instructors  these  Individuals 
would  form  a  "road  show"  instructor  team.  The 
teams  would  travel  to  each  SAC  base  and  conduct 
the  conversion  training.  They  would  provide  both 
the  ground  and  Inflight  Instruction. 

Due  to  assignment  priorities  for  fliers  and 
shortages  of  qualified  Instructors,  the  above 
concept  did  not  develop.  The  4235th  STS  and  SAC 
were  faced  with  the  problem  of  conducting 
extensive  training  with  minimum  Instructor 
resources.  A  new  concept  of  training  was 
developed  which  created  a  less  then  optimum 
situation.  The  responsibility  of  the  curriculum 
development  remained  with  the  4235th  STS  but  the 
responsibility  of  the  "road  show"  was  given  to 
the  4017th  Combat  Crew  Training  Squadron  (CCTS), 
Castle  AFB,  CA.  The  problem  arose  that  a 
training  program  was  being  produced  In  one 
organization  but  the  instructors  to  conduct  that 
training  would  come  from  another. 

A  three  step  solution  was  formulated  to 
eliminate  this  problem.  First,  a  channel  of 
communication  was  established  between  the  OAS 
project  officers  at  the  4017th  CCTS  and  the 
4235th  STS.  Boeing  documentation,  task  analysis, 
and  system  specifications  were  sent  to  the  4017th 
CCTS  to  establish  a  core  of  Information  for  the 
Initial  Instructor  team.  This  improved  the 
communications  between  the  two  organizations 
since  the  4017th  CCTS  now  knew  something  about 
the  OAS/ALCM. 

The  second  step  to  solve  the  problem  fell  on 
the  4235th  STS.  A  training  proyram  for  the 
Initial  "road  show"  Instructors  was  established. 
In  this  class,  the  4017th  CCTS  Instructors  would 
take  the  entire  conversion  course.  Doing  this, 
the  4235th  STS  was  able  to  perform  validation  on 
the  curriculum  and  the  4017th  CCTS  Instructors 
were  able  to  learn  the  system  plus  see  how  the 
course  was  produced  and  delivered,  4017th  CCTS 


Instructors  also  completed  a  course  in  ISO  so  the 
philosophy  of  the  courseware  was  more  meaningful 
to  them.  Both  organizations  became  confident 
that  the  other  was  doing  its  job. 

In  the  third  step,  the  4235th  STS  established 
a  procedure  to  have  at  least  one  COM  present  at 
each  base  for  the  entire  length  of  the 
conversion.  This  Individual  would  be  there  to 
oversee  the  training  program  and  to  make  note  of 
valid  changes.  He  will  not  be  an  Instructor  but 
an  evaluator  of  the  curriculum  presentation.  The 
4235th  STS  three  step  program  has  thus  far  been 
successful.  These  efforts  have  averted  a 
potentially  serious  situation.  However,  future 
programs  requiring  simultaneous  curriculum 
development  and  full-scale  development  should 
make  every  effort  possible  to  use  the  curriculum 
developers  as  the  Initial  cadre  of  Instructors. 
They  have  the  most  current  knowledge  of  system 
operation  and  courseware.  Serving  as  Instructors 
they  could  also  discover  changes  between  system 
operation  and  courseware  more  readily  due  to  the 
longer  amount  of  time  they  have  been  Involved  In 
the  system. 

CONCLUSION 

The  4235th  STS  has  produced  a  training 
program  simultaneously  with  full-scale 
development  of  the  system  requiring  the  training. 
Is  has  certainly  not  been  as  easy  as  conventional 
training  program  development;  however,  It  Is 
possible.  Evaluations  of  the  OAS/ALCM  training 
program  by  Strategic  Air  Command's  1st  Combat 
Evaluation  Group  and  the  Grlfflss  Air  Force  Bass 
staff  have  been  highly  complimentary.  With  the 
long  lead  times  required  to  bring 
todays' sophisticated  systems  Into  operation,  the 
combining  of  full-scale  development  and  training 
program  development  might  become  a  common 
occurrance.  Hopefully  the  suggestions  made  In 
this  paper  to  solve  the  typical  problems  of  such 
an  effort  will  help  In  the  devleopment  of  future 
training  programs. 
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THE  SIMULATOR  TRAINING  MATRIX 
Michael  P.  Scher 
Hughes  Aircraft  Company 

ABSTRACT 


There  are  five  basic  questions  that  should  be  answered  by  various  DOD  personnel  prior 
‘o  selection  and  procurement  of  a  training  device.  The  sequence  of  these  questions  and  who 
docs  the  answering  Is  a  critical  determinate  of  whether  or  not  the  ultimate  user  actually 
gets  what  he  wants  or  needs  to  fulfill  the  training  requirement.  An  example  of  a  simpli¬ 
fied  sequence  of  events  or  matrix  Is  Included,  which  can  be  applied  universally  to  all  new 
simulator  procurements,  modification  to  existing  devices  (CILOP  -  Conversion  In  Lieu  Of 
Procurement),  or  new  concepts  Incorporating  "non  training"  requirements. 


The  rapidly  changing  threat  environment  has 
created  a  technology  race  that  has  finally  reach¬ 
ed  a  point  In  which  the  human  factor  must  be  re¬ 
moved  from  mundane  operations.  New  weapon  sys¬ 
tems  being  delivered  have  attempted  to  accomplish 
this  by  Increasing  system  capability  and  complex¬ 
ity,  but  the  results  have  been  an  Increase  In 
the  operators  task  loading.  Although  the  manual 
manipulation  of  knobs  and  processing  of  Infor¬ 
mation  may  have  been  decreased  under  routine 
conditions,  the  assimilation  of  data  from  several 
sources  and  abnormal  situations  which  occur  only 
In  combat  scenarios  have  actually  Increased  the 
operator's  task.  To  properly  evaluate  the  equip¬ 
ment  within  the  limited  time  frame  of  combat  re¬ 
quires  thorough  knowledge  and  working  experience 
with  the  Individual  black  boxes  and  the  whole 
weapon  system.  This,  then,  sets  the  requirement 
for  a  training  program  capable  of  realistically 
reproducing  as  much  of  the  actual  combat  scoriarlc 
and  equipment  capabllltles/deflclencles  end  ab¬ 
normalities  (malfunctions)  as  possible. 

We  have  now  reached  the  point  where  the 
training  system  Is  necessarily  more  complex  than 
the  actual  prime  equipment  It  simulates.  At 
some  point  In  the  not  so  distant  past,  this 
phenomenon  would  have  resulted  In  Instant  turn¬ 
off  and  the  training  would  have  been  scaled  to  a 
point  less  than  necessary.  Fortunately,  the  In¬ 
creased  training  system  complexity  commanded 
more  budget  attention,  which  opened  the  doors  to 
more  professional  educators.  These  professionals 
then  proved  that  the  advantages  of  the  super 
training  system  far  off-set  the  heavy  Initial 
Investment. 

Many  people  have  asked  why  systems  are  not 
made  simpler,  requiring  less  training,  and  there¬ 
by  causing  less  budget  strain.  This  paper  will 
not  dispute  the  virtues  of  either  side,  but  only 
state  that  U.S.  defense  policy  Is  to  compete 
against  "numerical  superiority"  with  "qualitative 
superiority".  It  is  a  plain  and  simple  numbers 
game  where  they  have  the  manpower  numbers  In 
abundance  -  we  don't. 

The  problem  Is  to  supply  the  needed  training 
device  with  the  sophistication  required  within 
the  available  budget.  Since  funding  for  specific 
programs  of  significant  size  Is  either  funded 
within  the  prime  budget  or  assigned  a  separate 
line  number,  timing  of  the  requirement  can  be 
significant.  If  the  program  Is  new,  such  as  a 


new  aircraft  buy  or  modification,  the  funding 
level  asked  takes  trainers  (new  or  modifications) 
Into  consideration.  In  cases  where  modification 
Is  no  longer  applicable  or  the  trainer  Is  pro¬ 
cured  after  the  prime  Is  operational,  a  new  line 
number  Is  assigned  and  funding  must  be  justified 
based  on  the  merits  of  the  trainer  Itself. 

Maximizing  the  use  of  each  budget  dollar  Is 
easy  to  say,  but  difficult  to  accomplish.  Align¬ 
ment  of  priorities  within  the  DOD  budget  starts 
after  the  known  personnel  costs  are  deducted. 

What  Is  left  Is  hardware  and  RSD.  Training  Is  re¬ 
legated  to  the  bottom  rung  of  the  ladder.  This 
Is  where  training  dollar  utilization  Is  scrutin¬ 
ized.  This  Is  also  where  I  believe  the  Injustice 
Is  done.  A  system  that  uses  checks  and  balances 
enhances  the  probability  of  fair  and  Impartial 
decisions,  but  In  this  case  provides  for  waste  by 
contractor  and  military  alike.  No  one  can 
effectively  argue  that  good  training  does  not  add 
to  the  effectiveness  of  any  combat  system.  Re¬ 
gardless  of  the  weapon  system's  age  or  complexity, 
training  of  some  degree  (whether  operator,  main¬ 
tenance,  or  both)  will  add  to  Its  utility  and 
efficiency.  The  waste  comes  from  the  different 
directions  each  agency  takes.  In  each  service  we 
can  account  for  at  least  four  inputs;  the 
Washington  community  that  has  to  justify  the 
budget  figure;  the  using  Command  that  Inputs  Its 
priority  list;  the  procuring  Command  (NTEC,  PM- 
TRADE,  AFSC),  which  may  be  further  subdivided, 
and  the  user  who  has  to  actually  utilize  the  de¬ 
vice. 

The  breakdown  and  subsequent  confusion  re¬ 
sults  from  the  Importance  each  agency  attaches  to 
the  issues  relevant  to  the  requirement.  I  can 
think  of  five  basic  questions  which  are  pertinent 
to  any  requirement: 

1)  What  type  device  Is  best  to  satisfy  the 
requirement. 

2)  Sophistication/fidelity  required? 

3)  Number  of  devices  needed  for  best  return? 

4)  Value  of  the  trade-offs  (savings 
possible)? 

5)  New  concepts  avallablllty/advlseablllty? 

Each  of  these  questions  Is  asked  and  partial¬ 
ly  answered  by  at  least  one  of  the  directing 
agencies.  In  some  cases,  the  answers  will  cause 
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program  delays,  late  RFPs,  or  even  total  re¬ 
direction  of  the  program.  The  priority  given 
each  factor  is  the  key  Issue.  Who  should  control 
the  priority  assigned  Is  the  key  problem,  and 
just  as  significant,  where  should  the  compromise 
be  made? 

The  first  question  of  what  type  device  Is 
best  to  accomplish  the  objectives  appears  easy 
to  answer  for  the  uninitiated.  The  user  almost 
always  will  opt  for  the  most  elaborate  Innova¬ 
tive  new  contraption  "available".  Available  is 
an  Important  word  because  he  wants  It  now.  The 
budgeteer  attempts  to  use  the  existing  device 
with  a  minor  modification  effort.  This  Is  the 
Cl  LOP  (Conversion  In  Lieu  Of  Procurement)  prin¬ 
ciple  and  Is  a  very  effective  method  of  getting 
a  needed  training  capability  at  a  relatively 
Inexpensive  price  -  sometimes.  A  third  approach 
comes  from  the  engineering  faculty  that  tends  to 
push  technology  that  favors  an  innovative 
approach,  and  a  full  scale  development.  Neither 
of  the  three  methods  should  be  eliminated  but 
they  should  be  harnessed  Into  a  useable  matrix 
where  they  are  treated  fairly  In  accordance 
with  operational  priorities. 

Agreement  can  be  reached  on  the  correct  type 
of  device  for  most  situations  when  no  present  de¬ 
vice  exists  and  funds  are  either  limited  or 
capable  of  sustaining  the  latest  technology. 
Throwing  a  hitch  In  at  this  early  stage  is  the 
non-training  requirement.  This  elusive  non¬ 
training  related  capability  can  come  In  several 
flavors.  Another  Important  aspect  affecting 
relatively  large  programs  Is  politics.  Politics 
play  an  obvious  part  In  the  budget  cycle,  but 
often  forgotten  are  the  Issues  of  foreign  buys. 
State  favoritism,  and  reelectlon  commitments/ 
promises.  Images  and  morale  are  also  aspects  to 
be  considered. 

A  good  example  of  a  program  caught  In  the 
clutches  of  both  the  non-training  requirements 
and  the  conflict  between  agencies  Is  the  Air 
Force's  Companion  Trainer  Aircraft  (CTA)  program. 

Originally  heraloed  by  Strategic  Air 
Command  as  the  answer  to  several  near  and  far 
term  problems,  the  CTA  has  yet  (as  of  1  July 
1981)  to  be  clearly  defined  and  Into  a  contrac¬ 
tors  hands. 

Senator  Barry  Goldwater  spoke  at  the  1st 
Interservice/Industry  Training  Equipment  Confer¬ 
ence  In  1979  calling  the  CTA  program  "  .... 
an  Innovative  way  to  have  real  flying  training 
with  significant  fuel  and  dollar  savings."  The 
original  concept  did  that.  It  would  save  In  fuel 
by  providing  B- 52  crews  training  In  an  aircraft 
at  less  than  one-tenth  the  fuel  of  a  B-52, 
and  unlike  a  ground  trainer  It  provided  actual 
flight  training  that  could  be  judged  as  a 
positive  for  pilot  morale  as  well  as  proficiency. 

The  CTA  program  utilizes  an  off-the-shelf 
business  jet  with  both  real  and  simulated  equip¬ 
ment  In  the  passenger  compartment  to  train  a  B-52 
crew.  The  electronic  warfare  portion  was  to  be 
closed-loop  simulation  whereas  the  offensive  sys¬ 
tem  would  use  simulated  bombing  controls,  but 
real-time  radar. 

The  cockpit  would  receive  only  minor  Instru- 


ntatlon  changes  to  reflect  the  B-52  environment. 
Analyzing  the  CTA  program,  It  becomes  very  easy 
to  see  how  It  was  sidetracked  so  often.  The  prob¬ 
lems  started  with  Congress  were  aggravated  by 
the  contractors,  and  finally  ran  Into  Internal  Air 
Force  problems  related  to  solving  the  first  two 
conflicts.  Problems  from  Congress  came  In  funding 
profiles,  foreign  politics,  and  basic  civilian 
trust  of  the  mi’itary  objectives.  Normally  non¬ 
defense  contractors  jumped  In  early  to  exercise 
their  political  muscle  to  see  this  new  avenue  of 
potential  sales  start  up.  In  this  Innovative 
new  approach  to  training,  civilian  products  would 
find  a  relatively  large  market  not  previously 
open.  Finally,  the  user  (SAC)  and  the  buyer  (ASD) 
fought  over  the  requirements  and  procurement 
method. 

Back  to  the  type  of  device  to  be  utilized. 
Assuming  no  non-training  aspects  are  apparent,  the 
real  requirements  should  be  decided  by  the  user, 
then  negotiated  with  the  buyer.  Only  after  this 
procedure  Is  complete  and  fully  agreed  to  by  both 
parties  should  contractors  be  allowed  entry.  Now 
the  draft  RFP  and  Industry  comments.  Unfortunate¬ 
ly  It  never  seems  to  happen  this  way.  In  most 
cases  the  user  states  the  requirement  In  general 
terms  and  the  buying  engineers  attempt  to  design 
the  product  within  budget  constraints.  The  user 
does  not  object  strongly  to  the  procurement 
approach  or  the  specific  requirements  the  buyer 
described  In  the  RFP. 

With  this  scenario,  the  emphasis  Is  placed 
on  budgetary  constraints  and  how  the  service 
estimates  the  program  profile.  Whereas  this  may 
be  a  totally  realistic  approach.  It  Is  not  In  the 
best  Interest  of  the  service. 

Quite  often  the  cost  estimating  by  the  service 
Is  off  considerably  In  either  direction.  This  can 
serve  to  slow  down  or  kill  a  particular  project 
before  It  has  a  chance  to  beqln.  If  the  project 
Is  assumed  to  cost  more  than  the  budget  will  allow, 
the  requirements  may  be  cut  to  a  minimum.  This 
leads  to  a  mediocre  training  device  that  won't  do 
the  required  job,  but  the  user  has  no  choice  - 
take  this  Item  or  none  at  all.  When  the  RFP  Is 
issued  with  the  reduced  requirement.  Industry  will 
bid  the  budget  less  the  winning  price  strategy.  If 
the  original  requirement  had  been  pursued,  perhaps 
the  competitive  nature  of  our  system  would  have 
produced  the  project  ..Ithln  budget  by  Innovation. 

In  effect,  the  Government  is  robbing  Itself  and 
stifling  1nnovat1\e  competition. 

Back  to  the  five  basic  questions,  question  two 
Is  asking  for  a  qualification  of  the  sophistication 
or  fidelity  to  do  the  job.  Again  the  problem 
rests  with  who  should  answer  the  question.  En¬ 
gineering  can  certainly  Investigate  or  evaluate  the 
competitiveness  of  analog  versus  digital,  but  they 
should  not  have  the  final  ray  In  training  fidelity. 
This  should  be  answered  by  the  direct  user.  The 
direct  user  Is  not  the  using  Command.  It  Is  the 
simulator  supervisor  or  simulator  Instructor. 

These  are  the  individuals  who  can  answer  the 
question  of  device  requirements  better  than  any¬ 
one.  Let  the  Instructional  Systems  Development 
(ISPl  personnel  determine  the  training  goal  for 
the  device  and  the  simulator  supervisor  determine 
what  "he"  needs  to  achieve  that  goal. 
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The  third  question  Is  for  those  staffers 
that  have  finally  seen  the  "big  picture".  The 
number  of  devices  needed  Is  related  to  manning 
and  the  particular  operation.  This  question 
must  be  coordinated  between  the  user  and  buyer. 
The  user  can  state  the  number  of  places  the 
devices  will  be  required  and  the  time  required 
on  each  per  day.  The  buyer  can  utilize  this 
requirement,  combine  It  with  the  engineering 
assessment  of  life-cycle  cost  and  MTBF  (Mean 
Time  Between  Failure)  rates,  and  a  fairly 
accurate  number  can  be  arrived  at.  Although 
the  questions  of  number  Is  usually  answered  by 
a  pseudo-reliable  method,  problems  arise  In 
funding  profiles  and  force  structure  changes. 
Since  multi-year  funding  Is  probably  a  dream 
In  simulator  procurement  and  changing  ad¬ 
ministrations  bring  new  modernizing  Ideas  for 
the  military,  stability  In  numbers  will  pro¬ 
bably  be  no  more  accurate  In  the  future  than 
they  are  now. 

Evaluating  the  trade-offs  for  training 
(question  4)  can  be  hazardous  to  one's  career 
In  government  service.  Although  the  weapon 
systems  training  expert  may  evaluate  procurement 
timing  as  the  Important  factor  overriding  a 
cost  penalty,  the  final  result  may  not  even 
consider  the  device  availability.  Again,  as  In 
the  other  questions,  the  value  assigned  to 
specific  features  or  requirements  are  generally 
stipulated  too  late  in  the  game  (causing  them 
to  be  slanted  due  to  cost,  politics,  or  other 
known  Inputs)  and  by  the  people  least  affected 
by  the  outcome.  Up-front  assignment  of  values 
to  different  aspects  should  be  completed  and 
agreed  upon  at  the  same  time  the  requirements 
are  laid  out.  The  value  points  should  be 
assigned  by  requirement  priority,  rather  than 
realistic  expectations.  Too  often  cost  becomes 
predominate.  The  question  of  how  much  will  be 
allotted  to  spend  on  this  system  should  not  be 
utilized  as  a  criteria  for  establishing  re¬ 
quirements  or  priorities.  The  training  needs 
are  first.  Potential  savings  should  be  evaluat¬ 
ed  after  realistic  cost  and  performance  data  is 
evaluated  in  response  to  the  requirement. 

Looking  for  the  cheap  way  out  or  the  paper  sav¬ 
ings  that  everyone  claims  have  jeopardized  some 
training  programs  and  on  occasion  have  produced  a 
productof  little  actual  training  value  because 
the  program  structure  was  decided  before  the 
facts  were  In  or  the  requirements  were  defined. 

Many  times  the  contractors  will  bid  to  the 
budget  rather  than  to  the  requirements,  or 
even  worse,  bid  for  the  "buy-in"  with  the  ex¬ 
pectation  of  ECP's  (Engineering  Change  Pro¬ 
posals)  to  null  them  out  of  the  "red".  When 
this  occurs,  the  usual  result  Is  the  end  user 
gets  an  inferior  product  with  Idle  hopes  of 
recovery  far  down  stream.  Perhaps  the  answer 
is  a  complete  reversal  of  DOD  buying  strategy. 
Don't  set  a  budget  for  individual  Items,  but 
rather  a  total  figure  for  each  service  with 
further  breakdowns  for  such  categories  as  stra¬ 
tegic  or  tactical.  Issue  the  RFP's  for  projects 
deemed  worthwhile  on  a  priority  basis  each 
year  and  assign  budgets  to  the  programs  when 
the  proposals  are  evaluated.  Perhaps  a  tougher 
way  to  do  business,  maybe  impossible  to  convince 
Congress  (for  major  sums  of  such  projects  as 


MX,  CX,  Trident,  LRCA),  but  it  might  eliminate 
buy-ins  and  products  of  lesser  value  than  orig¬ 
inally  requested.  This  concept  will  seem  totally 
Irresponsible  if  not  closely  thought  over. 

Isn't  the  DOD  budget  already  decided  this  way? 

Real  growth  Is  projected  against  the  Inflation  and 
when  added  to  the  previous  years  figure,  a  new 
budget  is  born.  For  those  of  us  not  Involved  In 
the  budget  process,  isn't  it  odd  that  the  yearly 
budgets  always  Increase  by  a  small,  but  semi- 
predictable  growth  factor.  Can  anyone  remember 
when  the  DOD  budget  showed  radical  movement, 
either  up  or  down.  In  response  to  real  weapon 
system  costs  (except  possibly  during  actual  war 
periods)?  Don't  major  weapon  system  costs  get 
spread  ov6r  a  period  of  years  to  alleviate  major 
deviations  In  the  upward  straight-line  graph? 

Now,  without  changing  this  method,  let's  lump  all 
minor  cost  (relatively  speaking)  Items  such  as 
training  devices  Into  a  pot,  large  enough  to  com¬ 
pare  favorably  with  the  overall  budget  Increase, 
and  draw  our  Individual  allocation  by  program 
priority  after  the  costs  are  In.  Costs  are  now 
real  and  the  user  gets  the  required  capabilities 
on  his  "top  priority"  devices. 

Back  once-more  to  the  question  of  trade-offs 
or  savings,  the  question  today  Is  loaded  with 
problems.  Who  decides  the  trade-off  at  any 
point  during  the  program  acquisition  cycle?  The 
difference  between  a  full  visual  system  or 
motion  base  can  be  significant  in  cost,  but  the 
training  obtained  or  lost  by  elimination  may  be 
of  a  much  higher  value.  Negotiation  should  not 
have  to  occur  once  the  requirements  are  written. 

If  the  front-end  analysis  was  done  correctly  in 
the  beginning,  the  training  requirement  is  a  true 
requirement  and  should  not  be  reduced  or  eliminat¬ 
ed  to  achieve  cost  trade-offs. 

The  question  of  whether  to  proceed  or  not 
with  a  new  concept  should  be  answered  within  the 
engineering  faculty  of  the  procuring  agency.  They 
are  best  suited  to  decide  the  merits  and  risks 
in  a  new  approach  to  solving  the  requirements. 

This  question  must  not  be  asked  until  all  require¬ 
ments  are  defined,  costs  are  evaluated,  and  the 
procurement  timing  is  agreed  upon.  The  tendency 
for  engineering  to  explore  new  areas  and  concepts 
is  natural,  but  must  be  avoided  if  it  will  de¬ 
stroy  the  integrity  of  the  program  structure 
such  as  operational  vi«Mii*-'»  or  cost. 

These  brief  discussions  of  the  five  basic 
questions  are  incomplete  at  best.  A  full  answer 
to  each  question  would  take  an  entire  paper  in 
itself.  By  utilizing  a  matrix  of  decision  flow 
and  assigning  the  responsibility  for  the 
decision  points,  the  questions  are  put  in  pers¬ 
pective,-  thereby  eliminating  some  of  the  current 
problems  and  reducing  others  to  manageable 
levels.  The  crude  matrix  I  have  drawn  here  is 
elementary,  but  Is  should  serve  as  an  adequate 
example.  It  assumes  the  dimensions  enclosed  by 
the  five  questions,  and  the  relative  sequence  in 
which  the  decisions  should  be  made. 

Rather  than  following  the  matrix  point-by¬ 
point  to  the  conclusion,  I  will  ooint  out  only 
specific  areas  and  relationships.  Note  first 
that  funding  is  considered  last.  The  entire 
basis  of  my  paper  is  that  we  have  short-changed 
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ourselves  by  purchasing  the  wrong  Item  because 
It  "fit  the  budget  profile."  Define  the  product 
first,  then  budget  to  accomplish  the  task. 

The  Inclusion  of  a  non-training  requirement 
Is  a  significant  factor  normally  left  out.  Ob¬ 
viously  more  detail  could  be  placed  between  the 
decision  of  a  non-training  requirement  to  be 
fulfilled  and  the  resultant  buy  of  a  new  concept. 
CHOP  also  has  a  place  up  front.  This  block  Is 
where  the  Initial  "training"  requirement  should 
be  detailed  by  the  user.  Allow  the  user  command 
or  the  buyer  engineers  to  determine  the  applica¬ 
bility  of  the  CHOP  principle. 

The  rest  of  the  matrix  Involves  a  simplistic 
decision  tree  that  need  not  be  detailed  In  this 
paper.  The  necessary  factors  to  be  considered 
In  any  acquisition  cycle  are  listed;  the  -Order 
or  priority  and  the  decision  maker  are  the  Im¬ 
portant  concepts.  It  Is  the  obligation  of  both 
Industry  and  DOD  agencies  to  assure  the  public 
that  each  dollar  spent  Is  based  on  a  real  need. 
Purchasing  a  device  because  It  fits  the  budget 
does  not  guarantee  that  the  dollar  was  spent 
wisely.  Only  when  the  total  training  require¬ 
ment  Is  satisfied  and  an  Industry's  competitive 
nature  Is  exercised  Is  the  public  being  assured 
of  getting  Its  money's  worth. 
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ABSTRACT 


Determining  alternative  curriculum  sequences  is  a  tedious  task  involving  many  individuals 
and  analysis  of  large  amounts  of  curriculum-related  information.  Because  these  tasks  are  not 
readily  reduceable  to  mathematical  operations,  and  because  educators  and  curriculum  designers 
are  generally  not  so  inclined,  computer  intervention  into  this  design  process  has  been  meager. 
Nevertheless,  the  power  of  the  computer  to  handle  vast  amounts  of  information  coupled  with  its 
high  speed  manipulation  ability  makes  it  an  ideal  instrument  to  use  in  the  instructional  design 
process.  The  project  reported  herein  describes  the  development  and  application  of  a  model  by 
which  a  curriculum  may  be  analyzed  to  determine  alternative  instructional  sequences  based  upon 
curriculum  objectives  and  limiting  constraints.  The  project's  primary  goal  is  to  ultimately 
apply  the  model  to  the  analysis  and  design  of  instructional  sequences  for  16  closely  related 
courses  currently  under  development  by  the  U.S.  Navy  Recruiting  Command. 


THE  PROJECT 

The  U.S.  Navy  Recruiting  Command  has  under¬ 
taken  the  task  of  developing  training  programs 
for  16  individual,  yet  closely  related  jobs  (or 
billets)  within  the  duty  called  recruiting. 
Although  development  of  training  programs  is  not 
new  to  the  military,  the  approach  to  this  partic¬ 
ular  curriculum  design  problem  is.  Due  to  the 
close  interrelationship  between  and  among  each 
of  the  16  courses  under  development,  there  is 
heavy  reliance  upon  instructional  sequence.  For 
example,  several  competencies  have  been  identi¬ 
fied  which  overlap  in  many  of  the  16  courses. 
Because  such  overlaps  parallel  real-lit  recruit¬ 
ing  practices,  they  were  not  avoided.  It  is 
educationally  sound  practice  to  structure  student 
learning  experiences  so  as  to  simulate  reality  as 
much  as  possible.  Some  educational  psychologists 
believe  this  produces  maximum  learning  transfer. 

However,  the  payback  comes  in  the  form  of  an 
increased  demand  on  curriculum  sequencing.  A 
non-sequitor  or  ill-sequenced  curriculum  can 
damage  the  realism  of  learning  experience.  It 
can  also  reduce  the  student's  ability  to  internal¬ 
ize  the  concepts  presented.  The  student  may 
appear  to  perform  satisfactorily  in  the  school 
environment  but  become  disoriented  in  trying  to 
perform  a  similar  task  in  the  real-world  environ¬ 
ment.  Thus  the  need  for  well-sequenced  instruc¬ 
tion. 

The  cinii-icaZ  approach  to  determining  accept¬ 
able  instructional  sequences  has  characteristic¬ 
ally  been  human  intuition.  Such  an  approach  is 
time-consuming  in  that  it  seldom  produces  an 
adequate  sequence  on  the  first  attempt.  Addition¬ 
ally,  considerable  work  is  involved  with  each 
iteration.  In  this  current  project,  intuition  is 
simply  not  sufficient  for  the  task  of  aligning  16 
courses  into  a  unified  sequence. 

In  any  curriculum  design  problem,  there  are 
a  myriad  of  variables  which  may  dramatically 
affect  the  ultimate  instructional  sequence.  How¬ 


ever,  a  model  exists  in  the  literature  which  is 
capable  of  dealing  with  complex  systems  of 
interacting  variables. (1)  This  model,  known  as 
Interpretive  Structural  Modeling  (ISM),  has 
been  successfully  applied  to  the  sequencing  of 
process  elements  in  a  number  of  design  projects 
in  the  fields  of  engineering,  agriculture  as 
well  as  a  host  of  other  complex  scientific  and 
social  problems.  (2)  Unfortunately,  ISM  has  seen 
limited  use  in  the  field  of  education  as  a  tool 
for  planning  and  design.  In  fact,  this  writer 
has  found  only  one  such  use.  And  this  has  been 
accomplished  primarily  by  Sato  and  his  colleages 
in  Japan.  (3,4)  It  is  the  intent  of  this  project 
to  adapt  ISM  to  the  instructional  sequencing 
problem  and  build  upon  the  work  that  has  already 
been  done  in  this  area  with  the  hope  that  success¬ 
ful  development  here  may  spawn  more  educational 
uses  of  ISM  in  this  country. 

As  the  initial  project  report,  this  paper 
will  present  some  basic  theory  underlying  the  ISM 
concept  as  well  as  a  method  which  shows  great 
promise  in  assisting  the  curriculum  designer  in 
determining  appropriate  alternative  instructional 
sequences. 


COMPLEXITY  IN  THE  DESIGN  PROCESS 

The  instructional  systems  approach,  or  any 
systematic  approach  to  instructional  design  for 
that  matter,  is  anchored  in  mathematical  modeling. 
It  has  long  been  recognized  that  a  systems 
approach  to  instructional  development  is  patterned 
after  the  scientific  method(5)  which  is  in  itself 
a  modeling  approach.  (6)  The  question  then  arises 
as  to  why  the  design  of  instruction  is  not  treated 
by  a  mathematical  approach  to  approximating  the 
shape  and  scope  of  a  curriculum!  In  their  text, 
Programmed  Learning  in  Perspective,  the  authors 
allude  to  the  mathematical  character  of  curricu¬ 
lum.  They  describe  a  quasi-mathematical  technique 
(termed  the  matrix  technique)  which  is  useful  in 
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determining  optimum  unit  sequencing  within 
programmed  instructional  material.  (7)  Davies 
further  generalized  this  procedure,  demonstrating 
its  utility  in  optimizing  presentation  sequences 
for  objectives  of  an  entire  course  of  instruc¬ 
tion.  (8)  The  logical  extension  of  this  work 
leads  one  to  believe  there  may  be  a  method  by 
which  a  complex  curriculum  composed  of  disjointed 
competencies  might  be  alternatively  sequenced. 

Successful  instructional  design  models  call 
for  some  sort  of  determination  of  sequence  at  some 
time  during  the  design  process.  Often  this  is 
achieved  through  construction  of  objective  trees 
(or  hierarchies).  In  fact,  Instruction  in  the 
building  of  such  hierarchies  is  often  in  great 
detail  (9)  —  testimony  to  its  importance  in  the 
instructional  design  process.  To  anyone  familiar 
with  such  a  task,  it  is  immediately  obvious  that 
instructional  hierarchies  are  complex  structures 
not  only  to  build,  but  also  to  interpret.  The 
casual  observer  is  often  unable  to  visualize  the 
many  possible  sequencing  strategies  from  the  maze 
of  lines  displayed.  Such  insight  requires  a  know¬ 
ledge  of  the  course  content  and  at  least,  some 
grounding  in  basic  learning  psychology.  Vet,  even 
if  this  prior  knowledge  is  assumed,  the  task  of 
choosing  an  appropriate  sequence  from  all  the 
possible  sequences  displayed  on  the  hierarchy  is 
still  not  easy.  Mathematical  modeling  and  opera¬ 
tions  research  provide  some  interesting  algorithms, 
however,  which  demonstrate  the  potential  to  assist 
in  solving  complex  instructional  sequencing 
problems. 

In  their  paper  Unified  Program  Planning,  Hill 
and  Warfield  describe  a  method  for  reducing  complex 
systems  of  elements  (in  cur  case,  objectives)  into 
a  matrix  which  describes  their  mutual  relation¬ 
ships.  (10)  They  call  this  a  Azt^-^tUiAactLon 
matnix  because  it  contains  information  relating  to 
the  interaction  of  each  element  with  itself  and 
the  others  in  the  system.  The  authors  define  such 
a  matrix  as  containing  enough  information  to  con¬ 
struct  an  objectives  tree. 

For  this  project,  their  matrix  method  is  used 
in  developing  an  objectives  hierarchy  from  an 
initial  set  of  course  objectives.  The  worth  of 
this  matrix  method  is  in  its  ability  to  produce  a 
hierarchy  which  actually  contains  more  information 
than  hierarchies  developed  by  other  means.  As  an 


example  of  the  kinds  of  information  stored,  and 
generally  gleaned  from  typical  objectives  trees, 
consider  the  hierarchy  of  a  hypothetical  curricu¬ 
lum  containing  15  interrelated  objectives  as  shown 
in  Figure  1  below. 

Several  bit*  of  information  are  implicitly 
stored  in  this  hierarchy.  For  example,  OBJECTIVE 
1  appears  to  be  the  terminal  objective  for  the 
curriculum.  That  is,  all  other  objectives  either 
directly  or  indirectly  terminate  at  OBJECTIVE  1. 
Also,  OBJECTIVES  3,8,10,11,12,13,14,  and  15  are 
at  base  levels  with  no  supporting  objectives. 

Thus,  these  are  ideal  starting  points  for  sections 
or  modules  of  instruction.  Yet  another  hit  of 
information  available  from  the  hierarchy  is 
implied  by  the  arrows  connecting  the  various 
objectives.  Their  pattern  indicates  the  existence 
of  partitions  between  objective  clusters  (though 
such  partitions  are  purely  arbitrary) .  For  ex¬ 
ample,  one  such  partition  could  be  OBJECTIVES  11, 

6,  2,  and  1;  another,  OBJECTIVES  13,  12,  7,  2,  and 
1;  another,  OBJECTIVES  8,  4,  and  1;  still  another, 
OBJECTIVES  15,  14,  9,  5,  and  1;  etc.  Although 
such  partitions  are  arbitrary,  these  groupings 
give  some  indication  of  the  amount  of  information 
potentially  stored  in  an  objective  hierarchy.  All 
these  bxti  of  information  taken  together  represent 
a  detailed  picture  of  how  each  objective  interacts 
with  all  the  rest  in  this  particular  hypothetical 
curriculum. 

Yet,  a  completely  different  class  of  inter¬ 
actions  exists  which  also  come  to  bear  on  a 
curriculum.  This  class  contains  such  instruction- 
related  items  as  resource  constraints  (money,  man¬ 
power,  and  time),  student  needs,  types  of  learning 
activities  available  to  students  to  meet  course 
objectives,  types  and  timing  of  measurement  tests, 
etc.  Each  of  these  has  a  effect  on  whether  or  not 
a  given  instructional  sequence  will  work 
effectively.  However,  these  interactions  cannot 
be  stored  or  displayed  on  a  typical  objectives 
hierarchy,  such  as  that  in  Figure  1.  Even  by  look¬ 
ing  at  the  hierarchy,  it  is  impossible  to  discern 
if  such  interactions  were  taken  into  consideration 
in  the  hierarchy's  development.  Of  course,  this 
information  could  be  superimposed  onto  the  hier¬ 
archy,  however,  this  could  very  easily  complicate 
the  diagram  to  the  point  that  interpretation 
becomes  impossible.  The  reason  for  this  is  that 
there  seems  to  be  an  upper  limit  on  the  amount  of 


Figure  1.  An  objectives  hierarchy  containing  15  interrelated  objectives 
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Information  that  one  human  can  process  and  operate 
on  at  any  given  time.  "Research  tentatively  shows 
that  the  amount  of  Information  man  Is  capable  of 
processing  Is  limited,  and  more  data... do  not 
necessarily  increase  the  quality  of  decisions  ir. 
the  same  proportion."  (12)  It  must  be  made  clear 
at  this  juncture  that  the  self-interaction  matrix 
is  not  Intended  to  replace  the  objectives  tree,  but 
only  to  enhance  it.  "The  self-interaction  matrix 
...is  not  as  clear  as  the  objectives  tree  for  view¬ 
ing  the  relationships  among  objectives,  but  it 
incorporates  significant  advantages  in  relating 
objectives  to  constraints,  alterables  and  needs" 
inherent  in  the  instructional  system.  (13)  Thus, 
in  this  project,  both  the  matrix  and  the  objectives 
tree  are  utilized  to  their  maximum  advantages. 

In  actuality,  the  curriculum  designer,  the 
teaching  staff,  and  the  management  personnel  each 
recognize  a  different  set  of  such  interactions  as 
mentioned  above  which  impact  on  the  curriculum. 
Thus,  the  designer  must  spend  considerable  time 
with  teachers  to  develop  an  instructional  hierarchy 
which  takes  into  account  as  many  of  the  ancillary 
interactions  as  possible.  And  when  they  finally 
come  to  an  agreement  on  a  reasonable  teaching  se¬ 
quence,  they  may  find  (to  their  dismay)  that  the 
administration  rejects  the  plan  because  of  some 
constraining  factor  neither  the  designer  nor  the 
teachers  knew  about.  Such  situations  are  common 
and  illustrate  the  need  for  a  model  which  can  con¬ 
tain  and  process  much  more  curriculum-relevant 
information  than  is  currently  possible.  The  major 
requisites  of  such  a  model  would  have  to  be:  con¬ 
venience,  simplicity  and  utility. 

Convenience  can  be  described  as  the  ease  of 
applying  the  model  to  the  design  problem.  Simpli¬ 
city  refers  to  the  quantity  of  information  that 
must  be  provided  by  the  user  for  the  model's  oper¬ 
ation.  And  utility  can  be  expressed  as  the  model's 
adaptability  to  a  general  class  of  curriculum 
design  problems  -  from  the  relatively  simple  task 
of  sequencing  information  within  a  programmed  text 
to  the  highly  complex  task  of  determining  the 
sequence  for  effective  learning  in  a  ip-iAaZtd  "K 
through  12"  educational  network.  ISM,  the  model 
used  in  this  project,  poseases  these  primary 
requisites  in  varying  degrees  and  is  thus  a  likely 
candidate  for  the  curriculum  design  problem. 


CHARACTERISTICS  OF  A  BINARY  MATRIX 

Before  detailing  the  results  and  current 
status  of  the  project,  we  should  first  clarify  the 
terms  used.  The  literature  on  the  subject  is 
primarily  mathematical.  For  this  discussion,  the 
mathematics  have  been  simplified  in  some  places, 
and  eliminated  altogether  in  others.  In  its  place, 
intuitive  arguments  have  been  used.  Readers  inter¬ 
ested  in  the  actual  mathematical  derivations  are 
referred  to  the  work  of  Warfield.  (14) 

A  binary  matrix  is  a  square  array  of  elements 
whose  values  are  either  1  or  0.  If  all  the  ma in 
diagonal  elements  (from  upper  left  to  lower  right 
in  the  array)  are  lo,  the  matrix  is  said  to  be 
reflexive.  Thus,  an  irref lexive  matrix  has  some 
Os  on  its  main  diagonal.  An  irreflexive  matrix 
must  be  made  reflexive  in  order  to  be  analyzed  by 
the  matrix  method.  Fortunately,  this  is  easily 
accomplished  by  adding  to  the  irreflexive  matrix 


an  identity  matrix.  This  is  also  a  binary  matrix 
with  Is  along  the  main  diagonal  and  Os  everywhere 
else. 

The  rows  of  a  matrix  are  usually  referred  to 
by  the  letter  -i,  while  the  columns  are  usually 
referred  to  by  the  letter  /.  Every  matrix  element 
occupies  a  position  which  is  at  the  intersection 
of  a  row  and  column.  Thus,  any  arbitrary  element 
of  a  matrix  can  be  referred  to  as  the  (. Lrj )  ele¬ 
ment.  If  a  matrix  element  (i,j)  and  its  "mirrir- 
image"  element  (/,-£.)  are  the  same  value  (either  1 
or  0),  then  the  matrix  is  said  to  be  symmetric . 

The  degree  of  symmetry  depends  upon  how  many  ele¬ 
ments  (a.,  j)  are  matched  to  their  "mirror-images". 
To  illustrate  this  more  clearly,  note  the  mirror- 
image  quality  in  the  binary  matrix  in  Figure  2  on 
both  sides  of  the  main  diagonal.  For  clarity,  the 
zeros  have  been  removed 
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Figure  2.  Mirror- Image  Symmetry  Above  and  Below 
the  Main  Diagonal  (dashed) 

A  binary  matrix  may  have  a  few  assymetric  points 
and  still  be  considered  symmetric  for  purposes  of 
this  method  if  the  number  of  assymetric  points 
are  kept  to  a  minimum.  In  reality,  an  assymetric 
matrix  yields  the  best  instructional  hierarchy. 
Thus,  the  degree  of  assymetry  in  the  matrix  deter¬ 
mines  the  richness  of  the  resulting  hierarchy. 
However,  this  depends  upon  the  nature  of  the 
objectives  under  consideration  and  the  nature  of 
the  interactions  among  objectives  -  both  of  which 
are  dependent  on  the  type  of  curriculum  being 
designed. 


TRANSITIVE  RELATIONS  AND  DIRECTED  GRAPHS 

In  determining  an  appropriate  curriculum  se¬ 
quence,  considerable  thought  must  be  given  to  how 
each  instructional  object!1’  relates  to  all  other 
objectives  in  the  curricula....  During  the  so-called 
"front-end  analysis"  phase  of  a  design  project, 
relationships  between  what  the  student  needs  and 
what  the  curriculum  will  offer  to  meet  those  needs 
are  more  likely  to  be  philosophical  intuitions 
than  rigorous  proofs.  The  mathematical  character 
of  ISM,  however,  requires  a  more  detailed  analysis 
of  such  relationships.  These  relationships  are 
logical  rather  than  mathematical. 

Consider  the  logical  relationship  among  three 
objectives  (a,  b,  and  c)  as  illustrated  in  Figure 
3.  Figure  3A  shows  that  objective  a  relates  to 
objective  b,  and  that  b  relates  to  c.  However, 
objectives  fa  and  c  are  not  directly  related  to 
one  another.  Clearly,  if  objective  b  were  removed 
from  the  curriculum,  objectives  a  and  c  would  exist 
as  isolated  entities.  Such  a  relation  among  objec¬ 
tives  is  called  intransitive  because  there  is  no 
direct  relation  or,  or  connection,  between  objec¬ 
tives  a  and  £. 

Figure  3B,  on  the  other  hand.  Indicates  that 
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Figure  3.  Two  Types  of  Relationships 
Among  Objectives  a,  b  and  c 


all  objectives  are  directly  related  to  each  other. 
If  any  one  of  them  is  removed,  the  remaining  two 
are  still  linked  together  through  a  binding 
relationship.  A  transitive  relation  is  one  in 
which  each  objective  relates,  or  is  somehow  linked 
to  the  others  in  the  group. 

Though  we  have  used  the  term  "relation" 
numerous  times,  we  have  not  yet  clearly  defined  it 
A  relation  is  a  phrase  or  term  that  shows  how  two 
or  more  elements  (or  objectives)  interconnect,  or 
link,  to  one  another.  Whether  or  not  a  relation 
is  transitive  depends  not  so  much  on  what  relation 
is  used,  as  on  the  situation  in  which  it  is  used. 

For  example,  consider  the  relation  "is 
contained  within".  If  a  "is  contained  within"  b, 
and  if  b^  "is  contained  within"  c,  then  it  follows 
that  a  "is  contained  within"  c.  We  can  visualize 
this  relation  in  Figure  4.  Any  objectives  a,  b 


Figure  4.  Visualization  of  the  relation 
Is  contained  within 


and  c  for  which  this  relation  holds  true  is 
considered  a  transitive  set  of  objectives.  It  must 
be  borne  in  mind,  however,  that  even  though  the 
4 elation  is  transitive,  not  all  objectives  will 
suit  it.  If  one  particular  relation  is  not 
transitive  across  an  entire  set  of  objectives  under 
consideration,  a  relation  that  does  apply  must  be 
found.  Each  new  relation  chosen,  of  course,  must 
be  similarly  tested  to  insure  transitivity  within 
the  entire  objective  set. 

Some  relations  are  intransitive  in  all  but  the 
most  specific  of  situations.  For  example,  Warfield 
has  reported  that  the  relation  "obeys"  fails  the 
transitivity  test  (15):  if  a  "obeys"  Is,  and  if  b 
"obeys"  c,  a  may  not  necessarily  "obey"  c.  In  fact, 
most  relations  are  situation  specific.  They  must  be 
carefully  considered  in  the  context  of  the  entire 
objective  set. 


Once  a  transitive  relation  has  been  identi¬ 
fied,  it  remains  to  be  discovered  how  the  relation 
specifically  affects  each  pair  of  objectives. 

Does ,  for  example,  the  relation  link  objective  a 
to  objective  b,  or  vice  versa?  A  simple  example 
should  serve  to  illustrate  this  point.  Consider 
the  transitive  relation  depends  upon  p/ilo/i 
accomplishment  0(j.  if  objective  a  depends  upon 
pilot  accomplishment  oi  objective  b,  then  clearly, 
b  cannot  possibly  depend  upon  pilot  accomplishment 
0(5  objective  a.  In  addition  to  illustrating 
assymetry,  this  example  also  illustrates  the 
concept  of  directabllity.  In  the  above  example, 
an  arrow  could  be  drawn  between  objectives  a  and  b 
with  the  arrowhead  pointing  toward  objective  a  to 
show  that  a  depends  upon  pilot  accomplishment  oi 
b. 


If  all  such  directed  relations  between 
objectives  are  considered,  a  picture  of  the  inter¬ 
actions  can  be  obtained.  Such  a  picture  is  known 
as  a  directed  graph.  Warfield  has  shown  that  any 
directed  graph  or  Hlgnaph.  possesses  an  associated 
binary  matrix  (16).  A  given  binary  matrix,  how¬ 
ever,  may  produce  a  number  of  alternative  digraphs. 
Any  one  of  them  could  be  used  as  an  objectives 
hierarchy  to  describe  the  interrelationships  among 
instructional  objectives.  The  binary  matrix  needed 
to  produce  the  digraph  is  called  the  reachability 
matrix.  If  transitive  and  assymetric,  this  matrix 
can  be  manipulated  to  produce  a  digraph  (otherwise 
known  as  an  objectives  hierarchy).  The  procedure, 
described  by  Warfield,  requires  the  formation  of 
tables  consisting  of  various  arrangements  of  objec¬ 
tives  (17).  The  actual  procedure  followed  for  this 
project  will  be  described  in  greater  detail  in  the 
next  section.  This  cursory  overview  of  the  under¬ 
lying  theory  supporting  the  matrix  method  will 
suffice  for  our  purposes  here. 


THE  PROJECT'S  METHOD 

The  process  of  generating  a  digraph  from  a 
set  of  curriculum  objectives  is  a  straight¬ 
forward  approach  composed  of  the  following  steps: 

1.  Identify  the  objectives  of  the  curriculum. 

2.  Determine  a  transitive  relation  which 
applies  to  the  objectives  in  the  context 
of  the  instructional  situation. 

3.  Place  objective  relations  into  a  matrix 
format  -  termed  a  self -interact ion  matrix. 

4.  Manipulate  the  matrix  into  a  suitable 
form  -  termed  a  reachability  matrix. 

5 .  Re-order  the  rows  and  columns  of  the 
reachability  matrix  and  partition  it  to 
reflect  hierarchical  levels  -  termed  a 
modified  reachability  matrix. 

6.  Compute  a  hierarchy  (or  digraph)  from  the 
modified  reachability  matrix 

The  curriculum  design  project  described  in 
this  paper  follows  this  six  step  process  for 
generating  hierarchies  and  determining  instruction¬ 
al  sequences.  Since  the  approach  is  both  complex 
and  time  consuming,  computer  algorithms  have  been 
designed  to  perform  most  of  this  work.  The  remain¬ 
der  of  this  paper  details  the  process  followed  in 
the  Navy  curriculum  project. 
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After  the  front-end  analysis  had  been 
completed  for  the  16  courses  under  development,  a 
listing  of  tasks  required  for  training  were 
Identified.  And  from  these,  a  series  of  learning 
objectives  were  developed  for  each  course.  One 
course  was  used  for  the  pilot  study  lu  this 
project. 

Step  2 

The  transitive  relation  niCtiiCLKy  to 
acaomptLih  was  agreed  upon  by  the  subject  matter 
specialists,  the  curriculum  design  staff  and 
the  approving  board  for  curriculum  development. 
This  relation  was  used  in  the  analysis  of  the 
relationships  between  every  possible  pair  of 
objectives.  Since  18  objectives  were  originally 
identified  for  training  In  the  pilot  course, 

18  x  18  («324)  distinct  objective  pairs  were 
analyzed  via  the  agreed  upon  relation. 

Step  3 

For  each  of  the  324  objective  pairs,  a  1  was 
placed  Into  the  corresponding  cell  of  a  matrix, 

If  the  relation  was  true.  If,  however,  the 
relation  was  false  for  a  particular  pair,  a  0  was 
placed  in  the  appropriate  matrix  cell.  The 
resulting  matrix  required  approximately  four 
manhours  to  accomplish.  The  self-interaction 
matrix  which  resulted  is  shown  in  Figure  5. 

Objective  Number 
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Figure  5.  A  Self-Interaction  Matrix 
for  the  pilot  Course 


Warfield  describes  an  algorithm  with  which  a 
computer  can  be  programmed  to  accomplish  this 
data  entry  step  with  reduced  effort  on  the  part 
of  the  user. (18)  Currently,  the  algorithm  is 
being  modified  for  use  in  this  project,  but  was 
not  used  for  the  pilot  project.  After  creation 
of  the  self-interaction  matrix  of  Figure  5,  it 
was  loaded  into  a  BASIC  language  microprocessor 
via  a  prompting  routine  developed  by  Orwig.  The 
flowchart  of  this  routine  is  shown  in  Figure  6. 


Figure  6.  A  Prompting  Routine 
for  Matrix  Data  Entry 
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Objective  Number 


Tha  self-interaction  matrix  of  Figure  5  must 
be  manipulated  into  a  reachability  matrix  before 
further  analysis  can  be  performed.  This  manipu¬ 
lation  involves  raising  the  self-interaction 
matrix  (M)  to  successive  powers  (squared,  cubed, 
etc.  by  Boolean  multiplication)  until  the 
following  equality  is  met:  jjn  „  nn+',‘ 

algorithm  used  to  accomplish  this  multiplication 
process  is  presented  in  flowchart  form  in  Figure 
7.  (A  flowchart  of  the  entire  computer  program 
developed  by  the  authors  appears  in  the  Appendix.) 
According  to  theory,  if  there  are  N  objectives  in 
the  matrix,  the  reachability  matrix  will  be 
derived  in  f-1  or  less  iterations.  (19)  The  self¬ 
interaction  matrix  for  the  pilot  course  (Figure  8) 
was  converted  to  reachability  form  in  four  itera¬ 
tions.  In  other  words,  the  matrix  of  Figure  8 
multiplies  out  in  four  iterations  to  form  the 
reachability  matrix  in  Figure  9,  which  satisfies 
the  equality:  M3  , 
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Figure  7.  An  Algorithm  To  Convert 
A  Self-Interaction  Matrix  A(N,N)  Into 
A  Reachability  Matrix  PR(N,N) 


Figure  8.  The  Self-Interaction  Matrix  of  Figure  5 


Objective  Number 


12  3  4  5  6 


111111111 

89012345678 


Figure  9.  The  Reachability  Matrix  Derived 
From  the  Matrix  of  Figure  8 


Step  5 

The  purpose  of  this  step  is  to  partition  the 
reachability  matrix  into  submatrices  which  reflect 
the  levels  within  the  instructional  hierarchy. 

The  resulting  partitioned  matrix  will  be  the 
reachability  matrix  modified  by  row  and  column 
interchanges.  To  determine  the  eventual  order  of 
this  interchange,  a  table  is  created  which 
contains  a  reachability  set,  an  antecedent  set  and 
the  product  (or  intersection)  of  both  sets. 
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The  reachability  set  for  element  1  la  found 
by  Inspecting  row  1  of  the  reachability  matrix 
(Figure  9).  Every  1  In  row  1  corresponds  to  a 
column  Index,  and  every  such  column  Index  will  be 
In  the  reachability  set  of  element  1.  To  find  the 
antecedent  set  of  element  1,  Inspect  column  1.  To 
every  entry  of  1  in  column  1,  there  la  a  corres¬ 
ponding  row  Index;  and  the  set  of  such  row  Indices 
is  the  antecedent  set  of  column  1 .  Each  row  and 
column  is  similarly  considered  in  turn  thus 
producing  a  table  of  reachability  and  antecedent 
sets  for  each  row  of  the  matrix. 

In  Figure  9,  the  row  and  column  indices  (1-18) 
are  used  to  Identify  the  respective  elements  of 
the  reachability  and  antecedent  sets.  Tat  a  1  is 
constructed  from  Figure  9  by  inspection. 


Table  1.  A  Reachability  Table 


ROW 

1HDU(S) 

RUCKABILin  set  »<$> 

ANTECEDENT  9IT  4(8) 

IIT  moDVCT  ! 

8(S>n  4(S) 

l 

I  3  4  5  6  7  •  9  10 

II  12  1)  14  13  17  18 

1 

1 

2 

2  6  13 

2 

2 

) 

3  4  3  7  1  9  10  11 

12  13  16  13  17  1.9 

I  3  5  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  14  17  18 

4 

4  13 

1  2  3  *  3  7  8 

9  10  11  13  14 

»  ! 

)i  HMI0I1 

3  |  12  13  1*  13  13  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10  i 

11  13  14  17  18  i 

6 

6 

1  6 

6 

7 

8 

3  4  3  7  1  9  10  11 

12  13  14  13  17  11 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  14  17  18 

3  4  3  7  1  9  10  11 

12  13  14  13  17  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  14  17  11 

9 

3  4  3  7  8  f  10  11 

12  13  14  13  17  18 

1  3  3  7  8  9  10 

U  13  14  17  18 

3  3  7  6  9  10 

11  13  14  17  18 

10 

3  4  3  7  8  9  10  11 

12  13  14  IS  17  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10  ' 

11  13  14  17  18 

11 

3  4  3  7  6  9  10  11 

12  13  14  13  17  11 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  14  17  18 

12 

12  13 

1  3  3  7  8  9  10  11 

12  13  14  16  17  18 

12 

3  4  5  7  8  9  10  11 

12  -  14  13  17  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  14  17  18 

1  U 

3  4  3  7  8  9  10  It 

12  13  14  13  17  18 

I  3  5  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  14  17  18 

!  13 

13 

1  2  3  *  3  7  8  9  10  11 
12  13  14  13  16  >7  18 

13 

16 

12  13  16 

16 

16 

17 

3  4  3  7  8  i  10  It 

12  13  14  13  17  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

J  3  7  8  9  10 

1)  13  14  17  16 

11 

3  4  3  7  8  9  10  11 

12  13  14  13  17  18 

1  3  3  7  8  9  10 

U  13  14  17  18 

3  3  7  8  9  10  i 

11  13  14  17  16 

From  Table  1,  it  is  immediately  apparent  that 
the  only  rows  for  which  the  aet  product  equals  the 
reachability  set  are  rows  6  and  15.  These  two 
rows  are  therefore  removed  from  the  table  along 
with  all  references  to  numbers  6  and  15  everywhere 
else  in  the  table.  Thus,  rows  6  and  15  from  the 
reachability  matrix  (Figure  9)  become  the  first 
two  rows  of  the  modified  reachability  matrix. 

These  two  rows  will  be  considered  the  top  level  in 
the  instructional  hierarchy  (or  digraph). 

Ordinarily,  the  references  to  rows  6  and  15 
can  simply  be  erased  from  the  table,  and  the  next 
iteration  begun.  For  the  purpose  of  illustration 
in  this  paper,  however,  each  new  (reduced)  table 
will  be  enumerated. 

Removal  of  all  6s  and  15s  results  in  the 
reduced  form  of  Table  2.  This  time,  the 


Table  2.  Reduced  Table  -  Level  1  Removed 


ROW 

INDEX (8) 

RUCHAMUTT  8ET  R(8) 

ANTECEDENT  ItT  4(8) 

SET  MODUCT 
_8(1>A  AC*) 

1 

1 

1  3  4  3  7  1  9  10 

1)  II  13  14  17  11 

1 
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2 
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3  3  7  8  9  10 
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4 

4 

1  2  3  4  3  7  8 
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3  4  3  7  1  9  10  11 

12  13  14  13  17  10 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

II  13  14  17  18 

7 

3  4  3  7  8  9  10  11 

12  13  14  17  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  14  17  19 

• 

3  4  3  7  8  *  IQ  II 

12  13  14  17  18 

I  3  3  7  8  9  10 

II  13  14  17  19 

3  3  7  8  9  19 

11  1)  14  17  18 

9 

3  4  3  7  C  9  10  11 

12  13  14  17  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  14  17  18 

iu 

3  4  3  7  •  »  )u  11 

12  13  14  17  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  14  17  18 

11 

3  4  3  7  1  9  10  11 

12  13  14  17  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  9  9  10 

11  13  14  17  19 

12 

12 

1  3  3  7  8  9  10  II 

It  13  14  14  17  18 

12 

13 

3  4  3  7  8  9  10  11 

12  13  14  17  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  9  9  10 

11  13  14  17  18 

14 

3  4  3  7  8  9  10  11 

12  13  14  i:  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  U  17  18 

16 

12  14 

14 

14 

17 

3  4  3  7  1  9  10  11 

12  13  14  17  18 

1  3  3  7  8  9  10 

It  13  14  17  18 

3  I  7  9  9  10 

11  13  14  17  11 

18 

3  4  3  7  8  9  10  11 

12  13  14  17  18 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

11  13  14  17  16 

reachability  set  R(s)  and  set  product  columns 
match  for  rows  A  and  12.  As  before,  these  rowi 
are  removed  from  the  table  and  the  reachability 
matrix  to  become  the  second  level  in  the  modified 
matrix.  Again,  removing  all  references  to  4  and 
12  from  the  above  table  results  in  the  formation 
of  Table  3. 


Table  3.  Reduced  Table  -  Level  2  Removed 


ROW 

INDEX (S) 

RUCHAllim  SET  R(S) 

ANTECEDENT  SET  4(1) 

SET  M0 DUCT 
i<l>n  4(8) 

1 

13  3  7  9  9  10 

11  13  14  17  18 

1 

1 

2 

2 

2 

2 

3 

3  3  7  8  9  10  11 

13  14  17  18 

I  3  3  7  9  9  10 

II  13  14  17  U 

3  3  7  9  9  10 

11  13  14  17  14 

3 

3  3  7  9  9  10  11 

13  14  17  19 

I  3  3  7  9  9  10 

II  13  14  17  19 

3  3  7  8  9  10 

11  13  14  17  18 

7 

3  3  7  9  9  10  11 

13  14  17  18 

I  3  3  7  9  9  10 

II  13  14  17  19 

3  3  7  8  *  10 

11  13  14  17  18 

8 

3  3  7  9  9  10  11 

13  14  17  18 

I  3  3  7  9  9  10 

II  13  14  17  11 

3  3  7  8  9  10 

11  13  14  17  18 

9 

3  4  3  7  9  9  10  11 

12  13  14  13  17  19 

I  3  3  7  8  9  10 

II  13  14  17  19 

3  3  7  8  9  10 

11  IS  14  17  18 

10 

3  3  7  8  9  10  11 

13  1*  17  18 

I  3  3  7  9  9  10 

II  13  14  17  19 

3  3  7  8  9  10 

11  13  14  17  16 

11 

3  3  7  8  9  10  11 

13  14  17  19 

I  3  3  7  1  9  10 

II  13  14  17  19 

3  3  7  8  9  10 

11  13  14  17  18 

13 

3  3  7  9  9  10  11 

13  14  17  19 

I  3  3  7  9  9  10 

II  13  14  17  19 

3  3  7  8  9  10 

11  13  14  17  18 

14 

3  3  7  9  9  10  11 

13  14  17  19 

I  3  3  7  8  9  10 

II  13  14  17  18 

3  3  7  8  9  10 

II  13  14  17  18 

16 

14 

14 

14 

17 

3  3  7  8  9  10  II 

13  14  17  18 

I  3  3  7  8  9  10 

II  1)  14  17  18 

3  3  7  8  9  10 

11  13  14  P  18 

19 

3  3  7  9  9  10  11 

13  14  17  18 

I  3  3  7  8  9  10 

II  13  14  17  19 

3  3  7  1  9  10 

11  13  14  17  18 

From  Table  3,  the  third  level  of  the  modified 


matrix  is  shown  to  be  composed  of  rows  2,  16,  3,  5, 
7,  8,  9,  10,  11,  13,  14,  17,  and  18.  Deleting  all 
these  references  from  Table  3  results  in  the  forma¬ 
tion  of  Table  4.  Note  that  only  row  1  remains  to 
make  up  the  fourth  and  final  level  of  the  modified 
matrix.  The  resulting  modified  matrix  is  shown  in 
Figure  10.  The  heavy  black  squares  clarify  various 
submatrices  which  denote  the  four  levels  identified 
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from  Che  tablet-  Note  that  both  row  and 
column  deaignationa  In  the  modified  matrix  have 
been  Identically  interchanged.  This  is  automatic¬ 
ally  accomplished  by  the  computer  algorithm. 


Table  4.  Reduced  Table  -  Level  3  Removed 
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Figure  10.  The  Modified  Reachabilitv 
Matrix  Containing  Four  Hierarchical  Levels 

The  dashed  lines  within  the  level  3  submatrix 
identify  constituents,  or  interior  links,  within 
that  level.  The  largest  of  the  three  constituents 
is  called  a  universal  submatrix  because  it 
contains  all  Is  indicating  that  each  of  the 
associated  objectives  in  that  submatrix  are 
mutually  reachable  to  each  other.  In  the  litera¬ 
ture,  this  is  more  commonly  known  as  a  maximal 
cycle..  The  dashed  lines  to  the  left  of  each  of 
the  four  heavy-lined  submatrices  outline  what  we 
in  the  project  have  termed  communication  iub- 
maViic.U  which  essentially  describe  how  one  level 
communicates  with  the  level  above  it.  These 
submatrices  become  useful  in  determining  paths  in 
the  eventual  digraph. 


Step  6 


At  this  step,  all  required  information  exists 
in  the  modified  reachability  matrix  to  compute  the 
digraph.  Warfield  has  noted  that  a  given 
reachability  matrix  does  not  produce  a  unique 
digraph.  (20)  This  implies  that  more  than  one 
digraph  can  be  constructed  from  the  reachability 
matrix  of  Figure  10.  All  the  digraphs  constructed 
in  this  project  are  generalized  digraphs  which  are 
actually  composites  of  all  the  possible  digraphs 
contained  in  the  reachability  matrix. 

The  construction  method  is  illustrated  using 
Figure  10  for  reference.  The  following  illustra¬ 
tion  represents  the  process  used  to  manually 
construct  the  digraph  from  the  modified  matrix. 

The  computer  algorithm  accomplishes  this  entire 
process  in  a  manner  which  is  tAanipoAent  to  the 
user.  The  process  is  presented  here  for  those  who 
wish  to  develop  their  own  algorithms. 


Begin  by  laying  out  each  of  the  four  levele 
identified  by  the  heavy-lined  aubmatrlces  (levels) 
and  starting  at  the  bottom  of  the  matrix.  Level  4 
contains  only  row  1.  Level  3,  the  largest  level, 
contains  rows  2,  16,  3,  5,  7,  8,  9,  10,  11,  13,  14, 

17,  and  18.  Level  2  contains  rows  4  and  12.  And 
level  1,  the  highest  level,  contains  rows  IS  and  6. 
By  referring  to  the  dashed  submatrlces  (communica¬ 
tion  subuatrices)  to  the  left  of  each  level 
submatrix,  connecting  paths  between  the  objectives 
of  one  level  and  the  objectives  of  each  higher 
level  on  the  hleiarchy  can  be  determined. 

For  example,  the  level  4  communication  sub- 
matrix  (bottom  row  In  Figure  10)  has  the  following 
pattern:  (0  0  1  1  1  1  1  1  1  1  1  1  1) .  This 
pattern  matches  the  patterns  of  ruws  3,  5,  7,  8,  9, 
10,  11,  13,  14,  17,  and  18  in  the  thira  level. 

Thus,  a  connecting  path  from  objective  1  to  each 
of  those  mentioned  above  can  be  drawn  on  the 
digraph.  Note  here  that  no  connecting  path  exists 
between  objectives  1  and  16  or  1  and  2  because 
their  communication  patterns  do  not  match. 

On  level  3,  there  are  three  separate  parts 
(or  comtituenti)  within  the  level.  One  constitu¬ 
ent  ie  composed  of  objective  2;  another  is  composed 
of  objective  16;  and  the  third  is  composed  of 
objectives  3,  5,  7,  8,  9,  10,  11,  13,  14,  17,  and 

18.  As  stated  earlier,  this  third  constituent  is 
called  a  maximal  cycle.  Thus,  interconnection  paths 
can  be  drawn  on  the  digraph  between  objectives  3, 

5,  7,  8,  9,  10,  11,  13,  14,  17,  and  18. 

To  get  from  level  3  to  level  2,  the  level  3 
communication  submatrix  (the  dashed  matrix  to  the 
right  of  the  level  3  submatrix)  is  analyzed 
gainst  the  level  2  submatrix.  Since  level  3,  as 
was  shown,  possesses  three  separate  and  unique 
constituents,  there  are  three  unique  communication 
patterns  to  consider.  For  instance,  the  maximal 
cycle  constituent  (the  largest  within  level  3)  has 
a  communication  pattern  of  (1  1).  This  pattern 
matches  the  Is  in  both  row  12  and  row  4  of  level  2. 
Thus,  connection  paths  can  be  drawn  on  the  digraph 
from  any  member  of  the  level  3  constituent  to  each 
of  objective  12  and  4  in  level  2.  It  is  suggested 
here  that  only  one  menber  from  level  3  be  connected 
to  level  2  since  each  member  of  the  maximal  cycle 
is  already  connected  to  all  others  in  that 
constituent  (by  virtue  of  it  being  a  maximal  cycle 
set).  In  addition,  a  single  connecting  path  allows 
the  resulting  digraph  to  appear  considerably  more 
simple.  However,  the  choice  to  do  or  not  to  do 
this  is  completely  arbitrary.  Also  in  level  3,  the 
row  16  communication  pattern  (0  1)  matches  only  row 
12  in  level  2,  while  row  2's  communication  pattern 
matches  only  row  4.  Thus,  two  more  connecting 
paths  can  be  drawn.  Continuing  in  this  manner,  a 
complete  digraph  can  be  drawn  to  represent  the 
reachability  matrix.  The  finished  digraph  is 
shown  in  Figure  11. 

We  should  digress  here  for  a  moment  to  make  an 
important  point.  Tatsuoka  contends  that  a  digraph 
can  be  constructed  merely  by  analyzing  the  self¬ 
interaction  matrix  (he  terms  it  the  adjacency 
matAix) .  This  writer,  however,  believes  that 
although  Tatsuoka' s  contention  is  valid  and  logic¬ 
ally  consistent,  the  adjacency  matrix  contains  only 
enough  information  for  one  unique  digraph,  whereas, 
the  reachability  matrix  yields  a  more  generalized 
digraph.  In  a  manner  of  speaking,  the  reachability 
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Figure  11.  A  Digraph  for  the  Pilot  Curriculum 


Figure  12.  An  Alternative  Form  for  the  Digraph  of  Figure  11 
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matrix  is  a  composite  of  a  faml.  of  adjacency 
matricaa.  Thla  la  intuitively  true  since  the 
reachability  matrix  la  computed  by  ralaing  the 
adjacency  matrix  to  ccnaecutlve  powers. 

This  can  also  be  shown  mathematically.  Take, 
for  example,  the  number  10.  Thera  are  two  num'oera 
whose  consecutive  products  will  equal  16  -  they 
are,  of  course  2  (2x2x2x2)  and  4  (4x4).  Both 
consecutive  products  result  in  the  same  number  - 
16.  Howc<  er  the  original  numbers  2  and  4  are 
obviously  not  the  same.  This  same  analogy  trans¬ 
fers  to  thn  problem  of  whether  to  use  the  adja¬ 
cency  or  reachability  matrix  in  determining  a 
suitable  instructional  digraph, 

A  digraph  computed  from  the  adjacency  matrix 
will  undoubtedly  be  more  simplified  than  one 
derived  from  the  reachability  matrix,  although 
the  level  of  complexity  does  not  begin  to  become 
a  hindarance  until  very  large  numbers  of 
objectives  (40  or  more)  are  to  be  manipulated. 

In  other  words,  the  digraph  derived  from  the 
reachability  matrix  will  usually  contain  more 
paths  than  that  computed  from  the  adjacency 
matrix. 

Each  path  on  the  digraph  can  be  thought  of 
as  a  legitimate  transition  from  one  objective  to 
another  within  the  curriculum.  Looking  at  the 
digraph  in  this  way,  one  can  begin  to  see  that  by 
developing  such  a  transition- laden  digraph  yields 
a  more  fertile  data  base  from  which  alternative 
instructional  sequences  may  be  derived. 

In  the  pilot  project,  it  was  recognized  that 
the  digraph  of  Figure  11  could  be  redrawn  to 
yield  more  meaningful  information  to  the  curricu¬ 
lum  designer.  This  alternate  digraph  is  shown  in 
Figure  12.  This  type  of  digraph  is  called  a 
minimum  edge  fizfAuzntation  of  the  hierarchy  (22). 

Note  in  this  figure  that  the  maximal  cycle 
constituent  of  level  3  is  represented  by  a 
bi-directional  circle  Interlocked  via  objective  1. 
From  the  viewpoint  of  the  actual  course  curriculum 
there  is,  in  fact,  a  great  deal  of  coherence  among 
objectives  1,  3,  5,  7,  8,  9,  10,  11,  13,  14,  17 
and  18.  Thus,  it  is  not  coincidental  that  such 
a  pattern  has  emerged.  Note  also  that  objective 
6  can  only  be  reached  by  objective  1.  Therefore, 
any  instruction  concerning  objective  6  must  rely 
on  information  presented  during  instruction  on 
objective  1  -  if,  th«.t  is,  the  students  are  to 
see  a  logical  transition  from  one  lesson  to  the 
next.  Since  objectives  1  and  6  appear  isolated 
from  the  rest,  instruction  relating  to  these  two 
objectives  could  very  easily  form  a  module  o! 
instruction.  Indeed,  other  modules  begin  to 
emerge  from  the  digraph  upon  closer  inspection. 

It  will  be  left  to  the  reader  who  gains  pleasure 
from  such  activity  to  discover  these  other 
modules. 

This  in  and  of  itself  is  a  remarkable  tool 
for  the  curriculum  designer  -  to  be  able  to  iden¬ 
tify  "natural"  groupings  of  objectives  via 
mathematical  analysis.  However,  this  is  merely 
a  fringe  benefit  of  the  matrix  analysis  technique. 
As  the  computer  analyzes  the  reachability  matrix 
and  its  communication  patterns,  a  data  base  is 
formed  which  contains  ail  possible  legitimate 
transitions  from  any  given  objective  to  any  other. 
Once  computed,  this  data  base  is  used  for  compar- 


ion  with  •  user's  trsnsltion  selections. 

A  use’  esn,  in  fact,  experiment  with  various 
instructional  asquancaa  -  transitioning  from  one 
objective  to  another  until  an  entire  course  is 
created.  By  comparing  user-selected  transitions 
with  the  permissible  transitions  stored  in  Memory, 
the  computer  will  Inform  the  user  if  a  particular 
instructional  sequence  la,  or  is  not,  advisable. 

It  will  even  printout  the  sequence  created  by  the 
user  in  hard  copy,  if  a  printer  Is  attached. 

Figure  13  la  an  actual,  though  partial,  computer 
printout  of  the  interactive  instructional  sequence 
creation  routine. 
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Figure  13.  An  Interactive  Instructional  Sequence 
Dialog  Between  User  and  Computer 

LIMITATIONS  WITHIN  A  CURRICULAR  SYSTEM 

Naturally,  the  ultimate  decision  as  to  how  a 
curriculum  is  to  be  arranged  rests  with  the 
managers  or  administrators  of  the  curriculum.  It 
has  been  this  writer's  experience  that  a  major 
deficiency  of  front-end  analysis  is  the  inadequate 
attention  paid  to  the  interplay  among  the  numerous 
internal  and  external  constraints  and  limitations 
placed  upon  a  given  curriculum.  Limitations  such 
as  facilities,  personnel,  time,  money,  social 
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factor*,  ate.,  if  not  antlclpatad  in  advance  of 
aatabllahing  a  curriculum  aaquance,  could  raault 
in  the  ultimate  alteration  of  an  otherwise 
logical  Instructional  nequence. 

It  is  admittedly  a  complex  task  to  consider 
the  effects  of  all  possible  limitations  affecting 
a  curriculum  without  some  meana  to  organise  and 
manipulate  very  large  amounts  of  data.  The 
project  described  in  this  paper  has  illustrated  a 
method  with  the  power  to  expand  and  accomodate 
the  analysis  of  such  limitations  -  and  thus 
produce  an  ultimate  curriculum  sequence  which  is 
sensitive  to  those  limitations.  The  work  on  this 
expansion  forms  the  basis  for  Phase  II  of  this 
project  planned  to  be  completed  later  next  year. 

The  ultimate  goal  of  this  project  is  to 
develop  an  integrated  curriculum  for  16  closely 
related  courses.  Each  course  possesses  certain 
characteristic  limitations  which  are  either 
reinforced  or  overcome  by  the  remaining  courses. 

It  is  desired  that  this  project  will  produce  a 
curriculum  which  will  reconcile  the  majority  of 
those  limitations.  Such  a  goal  is  common  to 
curriculum  designs  both  in  the  military  and 
civilian  sectors  of  education.  In  that  respect, 
at  least,  those  of  us  associated  with  this  project 
feel  a  bond  with  educators  in  every  sector  of 
society. 
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A  Detailed  Computer  Flowchart  for  Developing 
A  Sequence  Digraph  From  a  Sat  of 
Curriculum  Objectives 
(continued  next  2  pages) 
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ABSTRACT 

A  Key  concept  in  evaluating  the  training  effectiveness  of  any  training  device  Is  the 
difference  between  its  use  to  achieve  positive  transfer  of  training  and  Its  use  to  evaluate 
student  performance.  As  simulators  are  becoming  more  realistic,  there  is  a  trend  towards 
using  them  not  only  to  reduce  training  on  the  operational  equipment  (e.g.,  aircraft),  but 
to  eliminate  entirely  all  training  and  evaluation  on  the  operational  equipment,  for  selec¬ 
ted  tasks.  In  order  to  eliminate  all  operational  equipment  training,  we  must  be  certain 
that  evaluations  of  student  performance  in  the  simulator  provide  a  valid  assessment  of 
actual  student  capabilities  on  the  operational  equipment.  The  assessment  of  the  validity 
of  student  evaluations  on  the  simulator  does  not  have  the  same  data  requirements  as  a 
transfer  of  training  study,  although  they  are  similar.  Unfortunately,  the  differences 
between  the  two  types  of  studies  have  been  virtually  unrecognized  by  the  training  device 
evaluation  community.  This  paper  will  discuss  the  theoretical  and  practical  differences 
between  the  two  types  of  studies,  differences  in  data  requirements,  and  an  example  of  the 
differences  which  were  observed  In  a  study  involving  the  KC-135  Boom  Operator  Part  Task 
Trainer  located  at  Castle  AFB,  CA. 


INTRODUCTION 

As  the  fidelity  of  training  simulators  has  in¬ 
creased,  the  use  of  simulators  for  evaluating 
trainee  proficiency  has  also  increased.  Previously, 
simulators  lacked  sufficient  face  validity  for 
training  system  managers  to  allow  final  trainee  per¬ 
formance  evaluations  to  be  conducted  in  them.  The 
simulators  did  not  look  or  feel  enough  like  the  ac¬ 
tual  equipment  for  most  tasks.  The  only  major  ex¬ 
ceptions  to  this  were  evaluations  of  emergency  pro¬ 
cedures  which  could  not  be  performed  on  the  actual 
equipment  for  safety  or  other  reasons.  Recent 
advances  in  simulation  technology  have  led  to  signif¬ 
icant  fidelity  improvements.  Now,  many  simulators 
look  and  feel  enough  like  the  real  thing  to  allow 
training  system  managers  to  consider  conducting 
trainee  evaluations  (e.g.,  checkrides)  on  the  simu¬ 
lators.  The  airlines  are  now  doing  this  on  a 
routine  basis. 

In  many  training  environments,  however,  there 
is  doubt  as  to  the  validity  of  an  evaluation  of  a 
trainee  conducted  in  a  simulator,  regardless  of  the 
fidelity.  "I  won't  believe  he  can  do  it  until  he 
does  it  in  the  real  thing,"  will  undoubtedly  be 
heard  more  than  once,  even  though  these  evaluations 
can  usually  be  conducted  much  more  efficiently  and 
inexpensively  in  the  simulators.  While  the  skep¬ 
ticism  is  healthy,  the  rest  of  the  sentiment  is  not. 
Face  validity,  in  terras  of  looks  and  feel,  is  not 
the  appropriate  assessment  technique  to  determine 
the  evaluative  capability  of  the  simulator. 


The  alternative  to  a  training  system  manager's 
intuition  la  empirical  data.  In  fact,  it  is  becom¬ 
ing  increasingly  common  to  collect  empirical  data 
for  many  aspects  of  training  effectiveness  (e.g., 
Hagln,  Osburne,  Hockenberger  and  Smith,  1980; 

Dltzlan  and  Laughery,  1979).  However,  most  of  the 
literature  pertaining  to  the  ways  and  means  of  con¬ 
ducting  training  effectiveness  studies  primarily 
address  the  measurement  of  transfer  of  simulator 
training  to  the  operational  equipment.  The  issue 
of  verifying  a  simulator's  adequacy  for  conducting 
performance  evaluations  has  rarely  been  addressed. 

If  the  data  collection  and  analysis  techniques  of  a 
simulator  transfer  of  training  study  automatically 
verified  the  validity  of  conducting  trainee  perfor¬ 
mance  evaluations,  there  would  be  no  problem.  Un¬ 
fortunately,  this  is  not  the  case.  While  there  Is 
commonality  between  the  transfer  study  and  valida¬ 
tion  study,  there  are  also  differences.  If  the 
transfer  study  is  not  specifically  designed  to  as¬ 
sess  the  validity  of  the  simulator  for  performance 
measurement,  it  may  not  be  possible  to  accurately 
assess  simulator  validity.  Therefore,  to  respond 
to  the  concerns  of  training  system  managers,  the 
theoretical  and  practical  issues  of  simulator 
measurement  validity  ought  to  be  addressed. 

The  remainder  of  this  paper  will  focus  on  these 
issues.  The  next  section  will  discuss  some  of  the 
theoretical  issues  of  evaluating  simulator  perfor¬ 
mance  measurement  validity.  A  central  focus  of  this 
section  will  be  to  highlight  the  differences  between 
evaluating  simulator  transfer  of  training  and  vali¬ 
dating  performance  measurement.  The  third  section 
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will  focus  upon  Che  practical  Issues.  These  are 
Issues  which  oust  be  considered  by  a  test  manager 
during  the  simulator  training  effectiveness  evalua¬ 
tion.  Finally,  the  fourth  section  will  present  a 
study  conducted  on  the  Boom  Operator  Fart  Task 
Trainer,  where  high  positive  transfer  of  training 
was  observed,  but  valid  simulator  performance 
measurement  capabilities  were  not.  This  example 
demonstrates  that  the  concepts  of  simulator  transfer 
and  validity  are  not  equivalent,  and  that  assump¬ 
tions  to  the  contrary  may  be  dangerous. 

THEORETICAL  ISSUES 

If  a  simulator  Is  to  be  used  as  a  predictor  of 
future  behavior  In  actual  equipment,  then  It  must 
be  a  valid  predictor.  This  means  that  we  must  be 
able  to  obtain  a  performance  score  from  the  simula¬ 
tor  and  use  it  to  predict  what  that  person' s  score 
will  be  on  the  actual  equipment.  The  simulator  may 
not  be  able  to  teach  the  subjects  a  thing  (although 
this  would  be  unlikely),  but  If  different  student 
simulator  scores  correctly  predict  different  student 
equipment  scores,  then  the  simulator  is  valid  as  a 
performance  measurement  device. 

If  the  simulator  is  to  be  used  as  a  training 
device,  however,  validity  is  not  necessary.  What 
must  be  demonstrated,  In  this  case,  Is  that  the 
simulator  produces  transfer  of  behavior  or  transfer 
of  training  (Caro,  1975).  Transfer  of  behavior 
simply  means  that  the  simulator  and  equipment  tasks 
are  Identical,  therefore,  the  student  does  not 
change  a  single  thing  when  he  goes  from  simulator 
to  aircraft.  Transfer  of  training  Implies  that  the 
simulator  will  permit  a  student  who  is  unskilled  at 
the  actual  tasks  to  obtain  these  skills  with  prac¬ 
tice  In  the  simulator.  Practice  In  the  simulator 
improves  performance  on  the  equipment,  but  the 
actual  behaviors  practiced  need  not  be  the  same. 
Transfer  of  training  Is  analogous  to  doing  calis¬ 
thenics  and  windaprlnts  to  improve  one's  football 
playing  skills. 

No  matter  what  the  measurement  validity  of  the 
device,  and  no  matter  what  the  training  simulator 
tasks.  If  the  student  group  using  the  simulator 
emerges  from  It  better  able  to  perform  actual  equip¬ 
ment  tasks,  transfer  of  training  occurred.  If  the 
relationship  (l.e.,  correlation)  between  simulator 
performance  and  actual  performance  Is  zero,  then  the 
simulator  is  not  a  valid  predictor,  but  It  still  may 
be  an  excellent  device  v+,en  viewed  from  the  transfer 
of  training  vantage  point.  To  recapitulate,  validi¬ 
ty  refers  to  the  device's  utility  In  performance 


prediction,  and  transfer  of  training  refers  to  the 
device's  utility  towards  facilitating  learning.  In 
operational  terns,  a  simulator's  ability  to  make 
valid  performance  predictions  relates  to  Its  utility 
In  assessing  terminal  performance  (e.g.,  checkrldes), 
and  its  transfer  of  training  utility  relates  to  the 
simulator's  ability  to  substitute  for  operational 
equipment  training  In  the  training  curriculum. 

These  concepts  can  also  be  perceived  graphical¬ 
ly.  In  evaluating  a  simulator's  validity,  we  are 
only  Interested  In  correlating  performance  In  the 
simulator  with  performance  on  the  actual  equipment, 
not  with  respect  to  learning  acquisition  rate.  The 
graph  in  Figure  1  describes  a  simulator  which  could 
serve  as  a  valid  predictor.  Transfer  of  training, 
on  the  other  hand,  need  not  Imply  a  relationship 
between  observed  simulator  performance  and  expected 
equipment  performance.  Figure  2  Identifies  a  situa¬ 
tion  where  transfer  occurs  without  the  simulator 
being  a  valid  performance  predictor.  The  top  graph 
describes  the  number  of  equipment  training  trials 
required  to  obtain  proficiency  without  prior  simu¬ 
lator  training.  The  second  graph  describes  the  num¬ 
ber  of  equipment  training  trials  required  subsequent 
to  training  in  r.  simulator.  The  third  graph  demon¬ 
strates  that  there  happens  to  be  a  lack  of  predictive 
validity  from  simulator  aircraft. 

In  this  case,  determining  transfer  of  training 
requires  careful  observation.  Clearly,  there  is 
transfer  of  simulator  training  to  the  actual  equip¬ 
ment,  In  that  far  fewer  equipment  training  trials 
were  required  to  achieve  standard  level  performance. 
However,  if  only  the  first  observed  equipment  per¬ 
formance  subsequent  to  simulator  training  were 
examined,  we  would  conclude  that  the  simulator  was 
Ineffective.  It  is  necessary  to  look  for  transfer 
on  more  than  the  first  equipment  trial.  The  rate  of 
learning  was  affected,  probably  due  to  the  learning 
of  basic,  skills  which  required  significant  refine¬ 
ment  to  be  useful  in  the  equipment,  but  did  not 
require  complete  relearning. 

Despite  the  difference  between  validity  and 
transfer  of  training,  the  concepts  derived  for 
measuring  the  former  are  often  applied  to  the  latter. 
A  number  of  approaches  to  assessing  transfer  of 
training  are  presented  by  Caro  (1975).  In  the  next 
section,  we  will  outline  the  basic  concepts  of 
validity  assessment,  and  Identify  Issues  which  need 
not  be  addressed  for  a  transfer  of  training 
assessment. 


Figure  1  EXAMPLE  OF  A  SIMULATOR  WHICH  HAS  PREDICTIVE  VALIDITY 
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PRACTICAL  ISSUES 

Considerations  During  Data  Collection 

There  are  two  considerations  to  address  during 
data  collection,  both  of  which  relate  to  the  measure¬ 
ment  of  student  performance  In  the  simulator.  First, 
trainee  performance  measurements  collected  in  the 
simulator  should  be  the  sane  as  those  collected  on 
the  operational  equipment,  in  order  to  test  whether 
observed  simulator  performance  is  highly  correlated 
with  observed  performance  on  the  operational  equip¬ 
ment.  If  there  Is  no  common  measure,  this  relation¬ 
ship  will  be  difficult  to  Identify.  Theoretically, 
we  can  correlate  two  variables  which  measure  differ¬ 
ent  dimensions,  like  age  and  height.  In  order  to 
create  intuitive  acceptance,  however,  it  is  best  to 
show  that  an  observed  level  of  trainee  performance 
on  the  simulator  implies  that  the  student  can  perform 


at  the  same  level  on  the  operational  equipment.  If 
we  do  not  use  the  same  measures  of  trainee  perfor¬ 
mance  on  the  simulator  and  operational  equipment,  it 
will  be  impossible  to  prove  that  simulator  and  oper¬ 
ational  equipment  performance  are  equal. 

The  second  consideration  is  when  to  measure 
trainee  performance.  In  a  transfer  study,  simulator 
performance  is  sometimes  ignored;  only  performance 
in  the  operational  equipment  is  measured.  However, 
the  only  way  to  obtain  an  estimate  of  the  degree  of 
correspondence  between  observed  trainee  simulator 
performance  and  expected  trainee  operational  equip¬ 
ment  performance,  is  to  compare  transfer  of  skills 
trained  in  the  simulator  to  first  trial  performance 
in  the  actual  equipment.  The  two  dimensions  to  be 
correlated  are  the  observed  performance  of  the 
trainee  the  last  time  he  performed  the  task  in  the 
simulator,  and  the  performance  of  the  trainee  the 
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first  time  he  performs  the  task  on  the  operational 
equlpaent.  If  a  high  correlation  Is  observed  between 
these  two  performances!  we  can  assuaa  that  the  simu¬ 
lator  Is  a  good  predictor  of  concurrent  performance 
In  the  actual  equipment.  Note  that  this  need  not  be 
his  final  performance  In  the  simulator,  only  his 
last  performance  prior  to  his  first  operational 
equipment  training  on  the  task.  In  fact,  there  may 
be  a  number  of  times  where  a  trainee  switches  from 
the  simulator  to  the  aircraft  and  then  back.  If  a 
portion  of  the  training  program  Involved  a  mix  of 
simulator  and  operational  equipment  training,  every 
time  the  student  switches  from  simulator  training  to 
operational  equipment  training  there  will  be  a  useful 
data  point.  We  measure  last  simulator  performance 
prior  to  the  next  set  of  actual  equipment  training 
trials,  and  first  performance  on  the  actual  equipment 
after  simulator  training.  However,  for  the  sake  of 
brevity,  we  will  use  the  terms  "last  simulator"  and 
"first  operational  equipment"  performance. 

Another  related  Issue  is  the  amount  of  time  be¬ 
tween  the  last  simulator  and  first  actual  equipment 
training  and  what  occurs  during  this  time  period. 
Basically,  the  shorter  the  time  period,  the  better. 
This  will  avoid  the  confounding  factor  of  forgetting, 
or  losing  the  skill.  Ideally,  we  would  measure 
trainee  simulator  performance  and  then  Immediately 
measure  trainee  performance  on  the  actual  equipment. 
This  will  usually  be  Impossible.  This  time  period 
should  be  minimized,  although  the  body  of  literature 
on  complex  skill  retention  Indicates  that  periods  of 
less  than  one  week  are  tolerable. 

During  the  period  between  last  simulator  trial 
and  first  actual  equipment  trial,  the  task  should 
not  be  practiced  or  reinforced  on  other  training 
devices.  Learning  or  skill  refinement  could  occur 
between  the  two  trials  during  which  performance  was 
measured.  That  being  the  case,  we  would  then  be 
correlating  simulator  performance  to  actual  equlpaent 
performance  plus  the  extra  learning.  This  extra 
learning  is  an  undesirable  confounding  factor.  How¬ 
ever,  when  the  trainee  is  measured  on  his  last  simu¬ 
lator  trial,  he  may  learn  from  this  trial  Itself, 

This  is  likely,  particularly  in  the  early  phases  of 
skill  acquisition,  where  the  rate  of  learning  per 
practice  trial  is  much  higher.  This  type  of  learning 
cannot  be  avoided.  It  may  be  possible  to  account  for 
this  learning  statistically  if  we  examine  the  learn¬ 
ing  curve  for  the  trainee  at  this  point  in  the  pro¬ 
gram,  His  simulator  performance  score  could  be 
increased  by  the  amount  of  improvement  which  would 
be  expected  from  that  trial.  If  the  measurements 
discriminate  among  levels  of  performances,  this  may 
be  possible.  However,  if  the  learning  rate  at  this 
point  in  training  is  reasonably  low  (e.g.,  less  than 
52  expected  improvement  per  trial),  or  our  measures 
are  not  sufficiently  discriminating,  we  may  ignore 
this  effect. 

Considerations  During  Data  Analysis 

The  correlation  to  be  computed  during  analysis 
is  that  between  last  trainee  simulator  performance 
and  first  trainee  actual  equipment  performance. 
However,  a  Bimple  correlation  analysis  is  not  enough. 
A  high  correlation  indicates  that  we  can  predict 
trainee  performance  on  the  actual  equipment  by  ob¬ 
serving  simulator  performance.  The  data  presented 
on  the  graph  in  Figure  3  indicate  a  high  correlation 
(above  .85),  However,  a  regression  analysis  would 
indicate  that  the  best  predictor  of  the  actual 
equipment  performance  grade  is  determined  as  follows: 


predicted  actual 
equipment  trainee 
grade 


o  k  j.  o  k  observed  simulator 

•  •  J  '  ws  J  X  .  ,  • 

trainee  grade 


In  this  example,  a  trainee  virtually  always 
performs  better  In  the  actual  equipment  than  on  the 
simulator.  This  could  be  because  of  simulator  con¬ 
trol  difficulties  which  are  not  present  on  the  actual 
equlpaent.  Therefore,  the  actual  equipment  Is  easier 
to  operate  than  the  simulator.  The  opposite  case  Is 
also  possible,  and  perhaps  more  likely  (l.e.,  actual 
equipment  trainee  performance  Is  always  something 
less  than  his  simulator  perforaance).  This  Is 
depicted  In  Figure  4. 

If  the  regression  equations  are  known,  and  a 
high  correlation  exists,  this  approach  to  predicting 
actual  equipment  trainee  performance  is  Indeed  valid 
and  useful.  However,  convincing  management  of  this 
may  be  another  story.  Someone  without  a  background 
in  statistical  analysis  may  have  a  difficult  time 
believing  that  the  simulator  ijs  a  good  predictor  of 
perforaance  only  if  we  take  the  score,  divide  it  by 
two,  and  add  2.5  to  that  value.  One  might  ask,  "If 
the  simulator  Is  such  a  good  Indicator  of  capabili¬ 
ties,  why  aren't  skill  levels  displayed  in  the  simu¬ 
lator  perfectly  representative  of  skill  levels  on 
the  actual  equlpaent?  Why  do  they  require  an  in¬ 
direct  functional  relationship?"  The  training  system 
manager  may  be  incredulous,  or,  at  worst,  disbeliev¬ 
ing  and  suspicious  of  the  study. 

There  is  no  simple  answer  to  the  problem  of 
knowing  an  answer  but  being  unable  to  convince  some¬ 
one  else  of  it.  Of  course,  if  the  means  and  standard 
deviations  of  the  actual  equlpaent  performance  and 
the  simulator  performance  are  Indeed  equal,  then 
using  the  raw  simulator  scores  to  predict  actual 
equlpaent  scores  presents  only  the  problem  of  regres¬ 
sion.  To  the  extent  that  correlation  is  less  than 
+1,  this  technique  fails  to  optimize  the  least 
squares  predictor  equation.  Studies  of  multiple 
regression  using  unit  weights  suggest  that  this  may 
be  less  of  a  practical  problem  than  it  appears  to  be 
theoretically. 

To  test  equality  of  means  and  standard  devia¬ 
tions,  we  find  ourselves  in  the  unenviable  position 
of  trying  to  accept  the  null  hypothesis.  One  way 
to  deal  with  this  is  to  set  up  confidence  intervals 
around  the  mean  and  standard  deviation  of  the  simu¬ 
lator  scores  (Hays,  1963)  and  compare  the  actual 
equlpaent  performance  means  and  standard  deviation 
to  these  intervals.  If  the  means  and  standard  devi¬ 
ations  were  identical,  and  the  correlation  were 
£•+1,  then  prediction  would  be  perfect  and  scores 
would  be  equal.  If  the  means  or  standard  deviations 
were  overly  different,  then  the  experimenter  should 
take  caution  before  recommending  the  use  of  raw  simu¬ 
lator  scores  to  predict  raw  actual  equipment  scores, 
regardless  of  the  value  of  the  correlation  coeffi¬ 
cient.  The  statistical  analyses  required  for  these 
decisions  go  beyond  our  present  scope.  In  fact,  It 
may  be  possible  to  derive  statistics  designed  to 
evaluate  these  problems  and  aid  in  decision  making. 

Now  let  us  present  an  example  of  a  training 
effectiveness  study  which  Included  an  examination 
of  the  utility  of  the  simulator  for  trainee  perfor¬ 
mance  assessment. 
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AN  EXAMPLE:  uiE  BOOM  OPERATOR  PART  TASK  TRAINER 
CATEGORY  QUALIFICATION  STUDY 


Boom  Operator  Part  Task  Trainer  ( BOPTT)  System 
Description  at  ckground 

The  KC-135A  BOPTT  Is  a  fixed-base  ground 
trainer  designed  to  duplicate  essential  air  refuel¬ 
ing  cues  used  by  the  boom  operator.  Major  compo¬ 
nents  of  the  BOPTT  are:  a  student  station  complete 
with  boom  operator  pallet,  window  operator  controls 
and  Indicators;  a  1/100  scale  model  of  the  B-52 
aircraft;  and  a  20  inch  model  of  an  aerial  refueling 
boom.  The  instruments  and  controls  operate  as  they 
would  in  actual  flight.  The  boom  operator’s  window 
Is  actually  an  optical  system  which  makes  the  boom 
and  receiver  aircraft  appear  as  they  do  in  the  real 
world. 

The  B-52  model  is  mounted  on  a  three-axis  gim- 
bal  that  simulates  aircraft  pitch,  roll,  and  yaw. 

A  v^deo  image  of  the  model  is  captured  by  a  closed 


circuit  TV  camera  and  displayed  on  a  CRT  screen 
placed  about  20  inches  outside  the  boom  operator's 
window.  Other  fighter;  and  cargo  aircraft  models 
are  also  available. 

The  model  boom  is  located  between  that  screer 
and  the  window,  and  Is  designed  so  that  the  trainee 
operator  can  position  and  extend  the  boom  and  can 
simulate  connecting  it  to  the  B-52.  Clouds  and 
ground  terrain  are  displayed  on  the  screen  as  they 
would  be  seen  from  the  rear  of  a  KC-135A.  Engine 
noise  and  boom  noise  are  produced  electronically 
and  played  through  speakers  into  the  boom  operator's 
station. 

The  BOPTT  portrays  the  B-52  or  other  aircraft 
as  it  approaches  the  tanker  from  1.25  miles  through 
hook-up  for  refueling;  the  boom  reacts  to  control 
inputs  and  aerodynamic  forces.  The  simulation 
equations  permit  variations  in  refueling  speed  and 
altitude,  amount  of  air  turbulence,  and  the  approach 
trajectory  of  the  receiver  aircraft.  It  is  also 
possible  to  simulate  five  levels  of  piloting  skill, 
which  range  from  novice  to  expert. 
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In  early  1980,  Strategic  Air  Command  (SAC) 
Headquarter*  conceived  that  the  BOPTT  night  prove 
to  be  an  effective  environment  In  which  category 
qualification  training  and  evaluation  could  be 
conducted.  The  BOPTT  had  been  In  uae  at  Caatle 
AFB,  CA  for  B-52  refueling  training  since  1978. 
Several  studies  (Cray,  1980;  Walker  and  Rutzebeck, 
1981)  Indicated  that  the  BOPTT  provided  a  highly 
effective  environment  for  Initial  training  of  boom 
operator  air  refueling  skills  for  several  receiver 
aircraft.  However,  In  the  Initial  training  program 
at  Castle  AFB,  most  training  and  all  checkridea 
were  performed  with  a  B-52  aircraft  receiver. 

There  are  significant  differences  In  skills  and 
techniques  required  of  the  boom  operator  to  refuel 
different  aircraft,  particularly  small  fighters 
(e.g.,  F-4,  F-16,  A-10)  as  contrasted  with  large 
tanker/cargo  aircraft  (e.g.,  C-5).  After  a  student 
completed  Initial  training  at  Castle  AFB,  he  could 
only  refuel  B-52  aircraft.  Subsequent  to  further 
aircraft  training  and  evaluation,  the  student  would 
become  category  qualified  (l.e.,  qualified  to  re¬ 
fuel  aircraft  from  other  categories).  The  other 
major  categories  most  frequently  encountered  were 
the  fighter  category  and  C-5  category.  Only  after 
a  student  wa3  evaluated  and  considered  qualified 
In  a  particular  category  could  he  then  refuel  the 
associated  aircraft  without  an  instructor  present 
in  the  KC-135.  It  was  proposed  that  the  BOPTT  be 
used  for  both  training  and  evaluation  for  C-5  and 
fighter  category  qualification.  Therefore,  both 
transfer  of  training  and  the  ability  of  the  BOPTT 
as  a  student  performance  assessment  device  were  of 
interest.  A  test  program  was  commenced  by  the  4200 
TES  and  the  93  BMW/D05,  both  located  at  Castle  AFB, 
to  evaluate  this  concept.  The  following  subsections 
discuss  this  study. 

Study  Design 

Two  phases  of  studies  were  conducted,  using 
boom  operator  students  as  subjects,  which  related 
to  evaluation  of  training  effectiveness.  The  first 
phase  was  conducted  at  Castle  AFB  and  the  second  at 
the  subjects'  home  stations.  The  first  phase  In¬ 
volved  the  training  on  the  BOPTT  for  C-5  and 
fighter  refueling.  After  the  subjects  had  been 
trained  on  the  BOPTT,  they  were  given  one  evaluation 
flight  each  for  refueling  fighters  and  C-5s.  The 
second  phase  Involved  an  evaluation  of  the  number 
of  actual  flights  required  for  qualification,  com¬ 
paring  control  (no  BOPTT  training)  and  experimental 
(BOPTT  trained)  groups. 


Subject* 

The  subject*  ware  30  initially  qualified  boon 
operators  from  the  Castle  training  program.  These 
subjects  were  derived  (5  each)  from  six  classes, 
during  the  period  July  through  December  1980.  It 
was  originally  planned  to  use  6  subjects  In  a  con¬ 
trol  group  and  24  in  an  experimental  group.  These 
subjects  were  selected  from  the  first  six  students 
of  each  class  assigned  to  the  fllghtllne  phase  of 
training  and  randomly  assigned  to  one  group  or  the 
other  for  the  study.  Upon  completion  of  the  first 
half  of  scheduled  testing,  the  decision  was  made  to 
increase  the  number  of  the  control  group,  In  order 
to  gain  a  more  accurate  assessment  of  transfer, 
particularly  In  fighter  air  refuelings.  The  final 
sample  consisted  of  10  subjects  in  the  control 
group  and  20  in  the  experimental  group. 

Procedure 

All  of  the  subjects  in  the  test  followed  the 
normal  training  syllabus  through  graduation  and 
solo  flight.  The  control  subjects  received  only  a 
categorization  qualification  briefing  prior  to  each 
flight,  In  accordance  with  SACM  51-135,  Vol  I.  The 
experimental  group  received  the  same  categorization 
briefing  plus  simulator  time,  which  served  as  a 
practice  medium  prior  to  each  flight.  The  experi¬ 
mental  group  received  two  one-hour  C-5  and  three 
one-hour  fighter  simulator  periods  In  the  BOPTT, 
Table  1  Illustrates  the  study  design. 

For  the  experimental  group,  the  existing  BOPTT 
mission  scenario  was  used  for  the  appropriate  cate¬ 
gory  aircraft  to  be  refueled.  The  93  BMW/DOTDK 
provided  the  necessary  BOPTT  missions  as  well  as 
the  Instructor  personnel.  A  90  minute  block  was 
used  for  each  training  period,  a  one-hour  refueling 
mission  profile  and  thirty  minutes  for  pre-  and 
post-mission  briefings.  As  a  result  of  some  diffi¬ 
culties  encountered  during  the  first  half  of  the 
test  (with  the  fighter  refueling  portion),  changes 
in  the  BOPTT  instructional  technique  were  Imple¬ 
mented  in  an  effort  to  overcome  problems  associated 
with  fighter  refuelings.  The  areas  of  boom  control, 
nozzle  cocking,  communications,  lack  of  confidence 
and  aggressiveness  were  specific  areas  of  concern. 
The  BOPTT  instruction  was  modified  to  Include  a 
demonstration  of  fighter  refueling  techniques,  the 
use  of  multiple  receivers,  and  creation  of  a  sense 
of  urgency  closer, to  £hat  found  in  the  real  world. 
The  task  difficulty, was  Increased  to  parallel  actual 


Table  1.  Study  Design:  Categorization  Research 


groups 

C-5 

CATEGORIZATION 

TRAINING 

EVALUATION 

F-4 

CATEGORIZATION 

TRAINING 

EVALUATION 

Control 
klO  Subjects) 

Categorization 

Briefing 

Actual  C-5 

Air  Refueling 

Categorization 

Briefing 

Actual  F-4 

Air  Refueling 

Experimental 
(20  Subjects) 

TWo  1-Hour 

BOPTT  Missions 

Actual  C-5 

Air  Refueling 

Three  1-Hour 
BOPTT  Missions 

Actual  F-4 

Air  Refueling 

Categorization 

Briefing 

Categorization 

Briefing 

Tir*1,1^1  -  • 
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or  wnrse  conditions  ns  would  be  encountered  in  the 
area*  of  rendetvous,  receiver  closure,  communlca- 
tions,  and  receiver  stability  prior  to  and  during 
contact.  Two  specially  constructad  objective  per¬ 
formance  tests  were  used  to  evaluate  the  tasks,  one 
for  the  C-S  end  the  other  for  fighter  refueling. 

These  tests,  collectively  named  the  Boom  Operator 
Category  Qualification  Performance  Measurement  Form, 
were  used  as  the  criterion  of  subject  ability  in 
tasks  relating  to  category  qualification.  The 
items  on  these  scales  were  taken  from  the  three 
skill  areas  (procedures,  communications,  and  boom 
control  and  operation)  critical  to  qualification. 

The  item  measures  were  either  quantitative  measures 
or  qualitative  ratings,  depending  on  the  item 
(task)  evaluated. 

Evaluation  Points 

Measures  of  training  effectiveness  for  the 
Experimental  Group  were  taken  at  each  simulator 
period  and  flight.  The  Control  Group  was  measured 
on  the  actual  flight  only.  The  Instructor/Evaluator 
graded  each  item  attempted.  These  evaluations  were 
then  used  to  arrive  at  an  overall  performance 
rating. 

Subsequent  to  their  departure  from  the  Castle 
AFB  training  environment,  test  data  from  the  sub¬ 
ject's  eventual  home  base  were  obtained  for  the 
entire  student  population  of  the  six  classes  in¬ 
volved  in  the  test.  The  data  were  gathered  by  a 
questionnaire  mailed  to  each  subject's  home  unit. 
This  questionnaire  provided  baseline  data  for 
making  a  more  accurate  assessment  of  number  of 
flights  required  to  qualify,  using  a  larger  control 
group  for  comparison.  The  home  unit  Instructors 
were  also  given  the  opportunity  to  exprese  their 
thoughts  on  the  categorisation  program  as  It  affect¬ 
ed  their  personnel.  Although  data  of  .  is  nature 
are  highly  subjective,  they  provide  Insight  that 
Is  otherwist  difficult  to  obtain.  The  opinions  of 
those  who  chose  to  answer  this  item  were  very 
positive.  These  instructors  generally  believed 
that  the  students  who  participated  in  the  program 
performed  better  than  the  students  who  did  not 
participate  in  the  categorization  training. 

Data  Analysis 

All  of  the  items  on  the  performance  measurement 
form  were  evaluated.  To  summarize,  BOPTT  trained 
individuals  did  significantly  better  than  those 
not  trained  in  the  BOPTT.  Of  great  interest  was 
the  question  of  whether  all  the  students  who  were 
graded  as  qualified  in  the  BOPTT  were  graded  as 
qualified  in  the  aircraft,  and  vice  versa.  This 
would  allow  the  BOPTT  to  be  used  to  measure  category 
performance.  Since  all  of  the  experimental  group 
students  had  been  graded  as  qualified  in  the  BOPTT, 


what  remained  was  to  determine  what  proportion  was 
qualified  in  the  aircraft. 

For  fighter  category  qualification,  the  follow' 
ing  data  are  presented  for  in-flight  performance i 


Considered 

Considered 

Qualified 

Unqualified 

Group 

In  Aircraft 

In  Aircraft 

Control 

2  (20%) 

8  (80%) 

Experimental 

10  (53%) 

9  (47%) 

Therefore,  even  though  all  19  experimental 
group  students  were  considered  qualified  in  the 
BOPTT,  only  53%  were  subsequently  considered 
qualified  in  the  aircraft.  Clearly,  this  was 
not  sufficient  to  consider  using  the  BOPTT  to 
evaluate  a  student's  ability  to  refuel  fighters. 
Almost  half  of  those  considered  qualified  in 
the  BOPTT  would  then  be  unable  to  perform  the 
task  In  the  air. 


The  data 

for  the  C-5  are 

as  follows: 

Considered 

Considered 

Group 

Qualified 

Unqualified 

Control 

8 

0 

Experimental 

13 

0 

Obviously,  since  all  students  were  con¬ 
sidered  qualified  in  the  actual  C-5  refueling, 
discrimination  was  impossible.  Any  B-52  quali¬ 
fied  boom  operator  la  apparently  capable  of 
performing  air  refueling  on  a  C-5. 

It  seems,  therefore,  that  the  BOPTT  was  not  a 
good  means  of  measuring  boom  operator  performance 
during  category  qualification  for  fighters.  The 
issue  remains  unresolved  for  C-5  qualification. 
However,  the  data  did  indicate  that  the  BOPTT  af¬ 
fected  transfer  of  training.  Phase  two  of  the 
study  clarified  this  issue. 

Performance  of  Subjects  at  Their  Home  Units 

The  second  phase  of  the  test  program  dealt 
with  the  performance  of  the  test  subjects  at  their 
home  units.  Questionnaires  were  sent  to  the  home 
units  to  obtain  information  regarding  the  categori¬ 
cal  status  of  the  subjects. 

Table  2  illustrates  the  number  of  flights  to 
qualification  and  the  percentage  of  each  group  for 
fighters: 


Table  2. _ Flights  Required  to  Qualify _ 

Groups _ One _ Two _ Three _ Four _ Five _ 

Control  3  (37. 5%)  1  (12.5X)  2  (25.01)  1  (12. 5%)  1  (12.51) 

Experimental  6  (42.85%)  6  (42.85%)  2  (14.3%) 

Other  (See  Note)  5  (17.9%)  11  (39.3%)  8  (28.6%)  2  (7.1%)  2  (7.1%) 

Note:  These  individuals  did  not  participate  in  the  test  program  but  were 
classmates  of  the  participating  members. 
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LEGEND 

.  EXPERIMENTAL 

-  CONTROL 

-  OTHER  (STUDENTS  NOT 

INVOLVED  IN  THE  STUOV) 


Figure  5  NUMBER  OF  FLIGHTS  TO  QUALIFY  VS.  CUMULATIVE  PERCENT  QUALIFIED 


Figure  5  implies  that  the  experimental  group 
attained  proficiency  more  rapidly  than  the  control 
and  other  (nonparticipant)  groups.  All  of  the  ex¬ 
perimental  participants  were  qualified  by  the  third 
flight.  It  is  interesting  to  note  that  approxi¬ 
mately  861  of  the  experimental  group  were  qualified 
by  the  second  flight,  as  opposed  to  57.21  of  the 
other  group  and  501  of  the  control  group.  As  in¬ 
dicated,  the  number  of  flights  to  qualification 
was  cut  by  approximately  one-third.  A  one-way 
ANOVA  on  the  flights  required  to  qualify  indicated 
a  significant  difference  among  the  three  treatments; 
J[(l,52)»3.95  £<.05.  Post  hoc  tests  indicated  that 
differences  existed  between  the  experimental  group 
and  both  the  control  and  other  groups.  The  control 
and  other  groups  were  not  found  to  differ.  There¬ 
fore,  a  substantial  savings  in  airframe  utilization, 
time,  and  money  can  be  realized  if  the  BOPTT  is 
adopted  formally  for  categorization  training.  These 
data  strongly  support  the  contention  that  the  BOPTT 
provides  training  which  does  transfer  in  a  positive 
fashion  to  the  aircraft. 

Discussion 

The  BOPTT  provided  positive  transfer-of- 
tralnlng  but  did  not  provide  a  good  environment  for 
student  performance  evaluation  with  respect  to 
category  qualification.  This  example  illustrates 
that  these  are  two  different  concepts  of  simulator 
use.  While  a  study  designed  to  evaluate  transfer 
may  be  modified  to  evaluate  validity,  one  does  not 
infer  the  other.  We  must  be  cautious  and  keep  these 
two  concepts  separate  to  ensure  the  safe,  effective 
use  of  simulators  in  the  future. 
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ABSTRACT 

A  theory  of  system  performance  measures  that  permits  evaluation  of  the 
effects  on  mission  performance  of  the  performance  of  each  constituent  mission 
task  was  extended  to  tactical  data  processing  systems.  This  system  performance 
measure  permits  the  evaluation  of  both  teams  and  Individual  team  members  on 
either  a  mission  or  any  portion  of  a  mission,  including  specific  tasks  or  task 
types. 


INTRODUCTION  from  a  search  of  the  literaturo  that  performance 

feedback  (knowledge  of  results  via  performance 
Organizations  both  civilian  and  military  measurement)  Is  critical  to  the  learning  of  team 

often  rely  on  teams  to  identify,  examine,  and  skills,  as  well  as  to  the  learning  of  Individual 

solve  complex  problems.  Some  teams  are  skills.  The  reviewers  strongly  warn  that  team 

formed  to  solve  short-term  problems  such  as  performance  measures  must  be  developed  before 

an  ad  hoc  committee,  and  other  teams  are  "substantial  improvement  in  team  training  evalu- 

formed  for  long  term  continuous  problems,  such  atlon  can  occur. "  To  support  this  conclusion, 

as  departments  of  research  and  development  or  they  cite  the  Defense  Science  Board's  1975(3) 

military  campaigns.  review  of  research  and  development  programs  In 

military  training  which  stated  that  "the  team  per- 
F or  any  particular  problem,  the  members  formance  measurement  problem  was  a  fLndament- 

of  a  team  bring  certain  skills,  viewpoints,  and  al  stumbling  block  to  progress  in  Improving  team 

limitations  that  are  wholly  individual,  yet  it  will  training." 

generally  be  from  the  interaction  of  the  members 

that  the  team's  effectiveness  and  worth  will  finally  A  research  program  suggested  by  Wagner  to 

result.  With  this  idea  in  mind,  it  is  significant  overcome  these  deficiencies  included  the  develop- 

that  previously  no  rigorous  method  existed  for  ment  of  a  method  for  establishing  team  performance 

evaluating  a  team's  performance  at  any  given  standards  for  both  the  isolated  and  interactive 

point  during  its  mission.  It  is  not  possible  to  behavior  of  team  members.  The  claim  that  inter— 

effectively  structure  teams,  train  teams,  or  to  active  behavior  produces  a  result  that  is  not  the 

specify  team  performance  levels  without  a  team  sum  of  individual  efforts,  but  is  rather  more 

performance  measure  and  a  systematic  means  like  the  product  of  them,  has  been  voiced  also  by 

for  applying  such  a  measure.  In  an  evaluation  Baker  (1976)(4)  and  by  Jones  (1974). (5)  Neglect 

of  Navy  team  training,  Rizzo  (1975)0)  states  of  this  proposal,  however,  has  led  to  an  emphasis 

that  basic  problems  exist  in  knowing  how  teams  in  research  on  individual  skill  training  at  the  ex¬ 
function,  that  teaming  skills  themselves  have  pense  of  research  on  team  training  and  on  co— 

been  left  undefined,  and  that  current  training  operative  behavior.  The  claim  is  that  an  effective 

methods  often  involve  throwing  individuals  to-  measure  of  team  performance  cannot  be  a  simple 

gether  with  the  hope  that  "the  needed  team  skills  summation  of  the  measures  of  performance  of 

will  naturally  emerge."  Also,  he  suggests  that  the  individual  members.  Indeed,  the  team  mea- 

team  performance,  as  it  is  envisioned  by  many  sure-of-performance  function  must  be  sensitive 

training  personnel,  is  related  more  often  to  how  to  such  non-linear  effects  as  the  quality  of  inter- 

well  individuals  know  each  other  than  to  what  action  among  them.  ~ " 

they  do  together.  Further,  he  stresses  the 

evident  lack  of  objective  measures  of  either  in-  EXISTING  METHODS  FOR  CALCULATING 

dividual  or  team  performance  during  training.  TEAM  PERFORMANCE 

A  review  of  the  state  of  the  art  of  team  train-  The  absence  of  team  performance  measures 

ing  and  evaluation  strategies  (Wagner,  1976)(2)  and  the  need  for  a  rigorous  method  of  developing 

confirms  the  lack  of  definition  of  teaming  skills  a  team  performance  measure  are  well  documented 

and  affirms  the  need  for  a  team  performance  in  the  literature.  Since  the  effective  interaction 

measure.  The  review  furthermore  concludes  of  team  members  is  regularly  cited  as  the  factor 
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that  distinguishes  an  effective  team  from  a  mere 
assembly  of  Ineffective  Individuals,  the  coordina¬ 
tion  of  the  members'  efforts  becomes  important 
both  to  teaching  and  to  maintaining  team  effec¬ 
tiveness  .  in  order  to  minimize  the  amount  of 
wasted  or  misdirected  effort,  the  amount  and 
quality  of  the  effort  produced  by  each  team 
member  to  enhance  In  some  (as  yet  unspecified) 
way  the  overall  task  effectiveness  or  "goal 
attainment . " 

On  certain  tasks  a  team  may  be  able  to 
operate  at  the  level  of  its  most  competent 
member  (a  disjunctive  mtxlel);  on  other  tasks, 
the  team's  performance  may  depend  upon  the 
effectiveness  of  its  least  proficient  member,  or 
even,  at  other  times,  on  the  level  of  its  "average" 
member.  But  whenever  the  efforts  of  tndls/iduals 
must  be  coordinated  for  task  accomplishment, 
there  is  inevitably  some  "slippage"  that  prevents 
optimal  task  accomplishment  by  the  team 
(Hackman&  Morris,  1975).(6)  As  team  size 
increases,  this  slippage  or  "process  loss"  be¬ 
comes  greater,  because  the  job  of  getting  all 
the  members  to  function  in  a  coordinated  manner 
becomes  increasingly  difficult.  When  Hackman 
and  Morris  claim  that  team  task  effectiveness 
generally  has  not  been  as  high  as  it  should  have 
been,  based  on  an  extrapolation  of  individual 
scores,  the  clear  implication  is  that  the  com¬ 
munication/coordination  process  through  which 
the  talents  of  team  members  are  first  assessed 
and  then  brought  to  bear  upon  a  given  task  must 
in  some  way  be  inadequate .  Such  process  losses 
may  be  insignificant  when  the  required  specific 
knowledge  or  skill  is  obvious,  or  when  obtaining 
a  desired  result  does  not  involve  complex  team¬ 
work;  but,  at  other  times,  when  sophisticated 
or  subtle  interactions  are  required  to  identify 
the  necessary  talents  and  apply  them  to  the  task, 
the  role  of  communication/coordination  becomes 
much  more  substantial,  and  the  risk  of  process 
loss  becomes  consequently  much  greater. 

Current  Military  Training  and  Testing 

Following  basic  training,  but  before  the 
new  soldier  goes  through  advanced  individual 
training,  his  aptitude  for  a  particular  Military 
Occupational  Speciality  (MOS)  is  evaluated  by 
means  of  the  Skill  Qualification  Test  (SQT). 

The  SQT  scores  for  an  individual  are  designed 
to  give  a  diagnostic  profile  of  the  examinee's 
strengths  and  weaknesses  in  a  given  MOS ,  and 
to  determine  whether  that  person  is  at  least 
minimally  qualified  to  perform  the  jobs  within  that 
MOS .  But  the  SQT  scores  do  not  predict  by 
themselves  how  well  that  person  will  perform  as 
a  team  member  whose  skills  must  be  integrated 
into  overall  team  performance.  The  current, 
and  still  evolving  Army  procedure  for  evaluating 
team  performance  -  that  of  a  tank  crew,  for  ex¬ 
ample,  or  a  platoon,  etc.  -  is  the  Army  Training 
and  Evaluation  program,  known  as  ARTEP. 


Problem*  of  Performance  Measurement  for 
Computerized  Systems 

For  clarification  we  must  distinguish  thrae 
important  entities  when  considering  the  perfor¬ 
mance  measurement  of  a  computerized  system. 
They  are:  1 ,  the  performance  measures  used, 

2 .  the  scores  that  are  obtained  from  using  the 
system  and  the  measure,  and  3.  later  tests 
:hat  may  be  performed  using  the  computerized 
system.  Here,  our  interest  is  in  the  develop¬ 
ment  of  a  performance  measure  for  teams  using 
a  computer  to  accomplish  a  goal,  and  within 
that  context  we  are  interested  specifically  in 
how  well  the  computer  equipment  is  being  used. 
Only  physical  variables,  such  as  the  time  used 
to  perform  a  given  task  or  an  indicator  of  the 
quality  of  the  task  performance,  are  involved, 
even  though  mental  effort  (selecting  a  data 
format,  for  example),  as  well  as  manual  effort 
fdata  entry  using  a  keyboard,  for  example), 
may  be  required  to  operate  the  system.  Since 
the  performance  measure  developed  here  is 
intended  to  reveal  only  how  well  equipment  is 
operated,  there  are  no  variables  that  correspond 
to  psychological  or  psychophystological  factors, 
such  as  intelligence  or  fatigue.  Nevertheless, 
the  system  (computer  equipment  and  team 
members)  along  with  its  performance  measure 
can  be  used  to  test  individual  and  team  pei — 
formance  levels,  to  evaluate  training  programs, 
and  to  determine  the  effects  of  operator  fatigue. 

Since  a  performance  measure  for  a  comput¬ 
erized  system  is  concerned  only  with  measuring 
how  well  equipment  is  used,  questions  generally 
associated  with  psychological  tests  (their  re¬ 
liability,  validity,  etc.)  are  not  of  immediate 
concern  to  us.  Only  later,  when  the  system  is 
tested,  will  they  be  of  direct  interest. 

Replacing  these  concerns,  however,  are 
two  others:  measure  comprehensiveness  and 
measure  sensitivity.  Measure  comprehensive¬ 
ness  is  evaluated  by  the  ability  of  the  measure 
to  respond  to  each  factor  that  affects  the  mission 
performance  of  the  system.  A  measure  with 
little  comprehensiveness  responds  to  only  some 
important  factors  and  is  often  referred  to  as  a 
"rule  of  thumb"  measure.  Measure  sensitivity 
is  evaluated  by  the  degree  to  which  the  measure 
reveals  the  effect  on  mission  performance  of 
changes  in  the  performance  of  individual  tasks 
or  types  of  tasks.  Measure  comprehensiveness 
and  measure  sensitivity  are  controlled  by  the 
way  tasks  are  defined  and  by  the  method  of  cal¬ 
culation  of  the  effect  on  mission  performance  of 
the  performance  of  each  constituent  task. 

The  fundamental  principle  to  be  observed 
in  the  performance  measurement  of  any  system 
is  that  the  performance  of  each  constituent  task 
always  has  a  unique  effect  on  total  misston  per¬ 
formance.  Even  the  same  type  of  task  performed 
at  different  timeo  during  the  mission  can  dlf- 
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ferently  affect  mission  performance.  For  in¬ 
stance  ,  a  data  entry  error  occurring  early  in  a 
mtssion  that  is  retained  in  the  local  computer 
memory  is  likely  to  be  easier  to  correct  than  a 
data  entry  error  occurring  later  in  the  mission 
and  distributed  to  many  units.  The  lateness  of 
the  data  entry  error  puts  a  time-stress  on  de¬ 
tecting  and  on  correcting  it,  since  wide  dispersal 
of  erroneous  data  increases  the  difficulty  of 
making  corrections.  Thus,  even  though  the  task 
(data  entry)  may  be  the  same  in  both  instances, 
its  eiTect  on  mission  performance  (e.g. ,  on  the 
probability  of  mission  success)  can  be  greatly 
varied . 

This  measurement  difficulty,  of  determining 
the  unique  effect  on  mission  performance  of  the 
performance  of  each  constituent  task,  leads 
frequently  to  two  types  of  errors  in  the  develop¬ 
ment  of  performance  measures .  The  most  com¬ 
mon  error  derives  from  the  false  assumption 
that  the  significance  of  the  level  of  performance 
of  a  given  type  of  task  is  independent  of  when 
that  task  is  performed  during  the  mission.  This 
results  in  the  specification  of  a  performance 
measure  which  considers  tasks  as  though  they 
were  performed  independently  rather  than  as 
coupled  components  of  a  total  mission. 

The  second  type  of  performance  measure¬ 
ment  error  derives  from  the  false  assumption 
that  the  varying  importance  of  task  performance 
can  be  accounted  for  by  a  single  scenario  which 
describes  a  fixed  sequence  of  mission  states, 
starting  from  the  initial  conditions  and  proceed¬ 
ing  to  the  final  objective .  The  error  here  is  in 
not  recognizing  that  once  the  actual  mission  has 
deviated  from  the  fixed  scenario,  the  prede¬ 
termined  task  performance  ratings  no  longer 
apply.  For  example,  consider  again  the  data 
entry  error  which  occurred  with  the  computer¬ 
ized  system .  The  team  that  is  now  attempting  to 
recover  from  that  error  may  be  demonstrating 
superior  error  recovery  performance,  but,  due 
to  the  fact  that  a  data  entry  error  was  not  in¬ 
cluded  in  their  scenario,  their  superior  error 
recovery  performance  cannot  bo  properly  rated 
or  utilized. 

Objectives  of  the  Paper 

The  objective  of  this  paper  is  to  present  a 
method  for  measuring  the  performance  of  teams 
using  computerized  systems.  The  measure  is 
to  be  a  function  of  the  performance  of  each  mis¬ 
sion  task,  including  any  interaction  among  team 
members  that  may  be  required  to  complete  the 
mission.  Also,  the  measure  shall  permit  the 
quantitative  evaluation  of  teams  as  units,  as 
wall  as  the  quantitative  evaluation  of  individual 
team  members. 

METHOD  OF  APPROACH 

The  theory  of  performance  measurement  in¬ 
troduced  by  Connelly,  Knoop,  Bourne,  & 


Loental  (1 969)C0  is  used  here  to  develop  a 
measure  foundation  of  the  overall  measurement 
performance  In  terms  of  the  Individual  task 
performance  effects.  This  theory  was  first 
applied  to  flight  control  problems  in  which  the 
factors  limiting  performance  originated  in  the 
hardware  and  were  known.  It  was  extended  for 
this  current  study  (Connelly,  Comeau,  & 
Stelnheiser,  In  press)(8)  to  permit  Its  applica¬ 
tion  to  team-computer  systems  where  the  factors 
limiting  performance  are  not  always  known  ex¬ 
plicitly,  but  are  known  to  exist. 

Since  the  factors  limiting  performance  are 
not  always  explicitly  known,  demonstrations  of 
tack  performance  at  various  levels  that  exhibit 
the  effects  of  those  limiting  factors  must  be 
used  in  developing  the  performance  measures . 
This  empirically  based  method  for  developing 
measures  is  described  by  Connelly,  Knoop, 
Bourne,  &  Loental  (1974)(9)  and  is  the  foundation 
of  a  computer  processor  known  as  MAP,  for 
Measurement  and  Analysis  of  Performance. 

MAP  extracts  information  from  the  performance 
demonstration  data  and  then  constructs  the  per¬ 
formance  measure, 

There  are  three  alternative  processes 
contained  in  MAP  for  constructing  the  measure, 
yet  it  is  known  that  one  process.  In  which  the 
mission  task  sequences  are  represented  by 
means  of  a  matrix  called  the  Transition  Matrix, 
provides  the  most  compact,  efficient  way  of 
computing  the  measurement  function  coefficients. 

In  order  to  describe  the  method  for  calculat¬ 
ing  a  team  performance  measure,  it  is  necessary 
to  first  define  two  types  of  tasks  and  two  types  of 
performance  measures. 

Classification  of  T asks 

The  goal-oriented,  or  "terminal,"  task 
begins  with  a  variety  of  initial  conditions  and 
ends  when  a  specified  objective  is  obtained. 

An  entire  mission  might  consist  of  multiple, 
sequential  tasks  for  which  the  terminal  condition 
of  one  task  is  the  initial  condition  for  a  subse¬ 
quent  task.  The  point  is,  that  with  terminal 
tasks  there  is  t.lways  a  specific  goal  to  be 
achieved,  such  that  when  the  goal  has  been 
achieved,  the  task  ends. 

Continuing  tasks,  on  the  other  hand,  have  no 
end  objective,  but  instead  require  performance 
specified  by  certain  criteria  at  each  instant  of 
time.  For  example,  the  well-known  pursuit 
tracking  tasks  used  in  psychological  studies 
are  continuous  inasmuch  as  the  participant  must 
constantly  manipulate  a  control  device  to  track 
a  moving  reference  point  in  an  attempt  to  keep 
his  error  as  small  as  possible  over  the  total 
test  time.  Typically,  the  error  Is  the  distance 
between  a  moving  reference  symbol  (such  as  an 
"x")  and  a  tracking  symbol  (such  as  an  "o"), 
controlled  by  the  participant.  The  test  Is  con- 
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ducted  for  as  long  as  the  experimenter  has 
planned,  and  a  performance  score  Is  developed 
as  the  average  error  over  the  test. 

Many  applied  human  factor  problems  can 
be  cast  as  terminal  control  problems,  even 
some  that  are  spoken  of  as  continuous  tracking 
tasks.  For  Instance,  In  the  sighting  of  anti¬ 
aircraft  guns  the  term  "tracking  the  target" 

Is  often  used.  But  this  problem  can  also  be 
broken  down  Into  a  sequence  of  terminal, 
goal-oriented,  tasks.  First,  the  operator 
attempts  to  acquire  the  target  in  his  sight,  a 
task  which  requires  the  reduction  of  large 
errors  by  the  operator.  Then,  once  the  target 
has  been  acquired,  the  operator  attempts  to 
track  it  smoothly  and  with  sufficient  lead  to  per¬ 
mit  a  hit.  If  automatic  lead  prediction  circuits 
are  available,  the  operator  must  still  continue 
to  track  the  target  smoothly  until  the  initial 
transients  in  the  prediction  circuits  can  die  out 
and  the  tracking  aids  can  calculate  accurate  pre¬ 
diction.  In  either  case,  the  operator  must  next 
commence  firing  and,  if  tracers  are  used,  must 
adjust  his  tracking  to  make  use  of  the  tracer  in¬ 
formation.  Finally,  when  a  hit  is  scored,  or  the 
enemy  aircraft  moves  out  of  range,  the  tracking 
task  ends. 

Still  other  tasks  may  be  viewed  as  either 
continuous  or  as  terminal.  Thus,  for  example, 
maintaining  aircraft  altitude  and  heading 
over  a  long  period  of  time,  such  as  in  the 
constant-altitude  cruising  phase  of  a  lengthy 
flight,  could  be  viewed  as  either  a  continuous 
tracking  task  or  as  a  terminal  problem,  de¬ 
pending  on  the  availability  of  a  relief  pilot  or 
of  an  autopilot,  among  several  possibilities. 

It  is  only  when  the  fundamental  purpose  of  a 
task  is  the  achievement  of  well  defined  final 
conditions  that  the  task  (or  mission)  must  be 
considered  terminal. 

Types  of  Performance  Measures 

Summary  Performance  Measures .  A  sum¬ 
mary  performance  measure  (SUMPM)  is  a  set 
of  rules  for  scoring  each  mission  exercise, 

(Note  that  in  order  to  describe  the  measure  it  is 
necessary  to  use  two  terms:  "mission"  and 
"exercise."  A  mission  is  the  set  of  tasks  that 
must  be  completed  to  accomplish  a  goal.  An 
exercise  is  one  demonstration  of  the  mission.) 

A  SUMPM  provides  measurement  only  of  the 
total  mission  performance,  and,  as  a  result, 
the  complete  information  required  for  a 
SUMPM  is  not  available  until  the  exercise  has 
been  completed.  This  property  is  a  fundamental 
limitation  of  all  SUMPM's. 

Typically,  SUMPM's  are  first  formulated 
subjectively,  and  reflect  the  judgment  of  an 
individual  or  group  concerning  the  objective 
of  the  mission  and  the  factors  believed  to  be 
important  in  scoring  exercises .  These  factors 


may  involve,  for  example,  statements  about  cer¬ 
tain  desired  terminal  and  safety  conditions  that 
must  be  satisfied  by  the  exercise.  But  whatever 
the  factors  are,  the  subjective  form  of  the  SUMPM 
must  then  be  converted  into  a  quantitative  form  in 
which  specific  rules  determine  the  SUMPM  value 
from  the  exercise  data. 

In  many  studies,  performance  measurement 
development  is  terminated  at  the  summary  level, 
even  though  SUMPM's  cannot  provide  sensitive 
performance  discriminations,  nor  reveal  the 
effect  of  individual  and  team  technique  on  task 
performance. 

System  Performance  Measures.  The 
theoretical  development  of  a  system  performance 
measure  (SYSPM)  which  reveals  the  effect  of 
the  performance  of  each  constituent  mission  task 
on  summary  performance,  and,  as  a  result, 
provides  sensitive  performance  discriminations. 
This  theory, which  was  developed  by  Connelly, 
Zeskind,  &  Chubb  (1977),  (10)  recognizes  that 
performance  is  limited  both  by  machine  factors 
and  by  human  factors .  Recognition  that  such 
limiting  factors  exist,  whether  or  not  they  are 
explicitly  known,  leads  to  a  measurement  equation 
that  permits  evaluation  of  the  effect  oi  either 
instantaneous  or  of  interval  performance  on  the 
performance  of  the  entire  mission.  The  theory 
has  been  successfully  applied  to  aircraft  and  ship 
control  problems  (Connelly,  1977), 01)  and  is 
applied  here  for  the  first  time  to  team-computer 
systems . 

Once  having  selected  a  particular  SUMPM  - 
that  set  of  rules  used  for  scoring  each  (necessarily 
completed)  mission  exercise  -  the  SYSPM  re¬ 
lates  in  mathematical  terms  the  effect  of  the 
performance  of  each  constituent  mission  task 
on  the  SUMPM  chosen.  With  the  SYSPM,  the 
effect  on  mission  summary  performance  of  the 
way  each  constituent  task  is  performed  can  thus 
be  assessed.  This  is  an  important  property 
since  the  effect  of  operator  task  performance  can¬ 
not  be  expected  to  be  uniform  over  all  team- 
computer  system  states.  The  SYSPM  function  has 
also  the  further  ability  of  being  able  to  discriminate 
among  the  many  ways  both  good  and  bad  team 
performance  can  be  achieved.  And,  since  team 
members  can  and  do  cooperate  in  various  ways 
to  achieve  high  performance,  this  property 
becomes  important  when  measuring  the  per¬ 
formance  of  teams  that  are  to  be  compared. 

To  obtain  these  properties,  the  SYSPM 
function  utilizes  "reference-task  performance" 
and,  in  addition,  the  effect  on  the  summary  per¬ 
formance  of  deviations  from  reference-task  per¬ 
formance  .  A  reference-task  performance  is 
defined  here  as  an  established  way  of  performing 
a  particular  task.  It  may  include,  for  example, 
the  time  required  to  complete  the  task,  the 
number  of  errors  permitted  in  attempting  the 
task,  and  so  on.  A  reference-task  performance  is 
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simply  the  established,  though  not  necessarily  the 
best,  way  to  complete  the  given  task. 

SYSPM's  provide  a  sensitive  and  compre¬ 
hensive  performance  measure  for  tasks  and 
mission  segments.  By  utilizing  reference-task 
performance  and  any  significant  deviation  from 
such  performance,  SYSPM's  provide  Information 
that  enables  them  to  identify  critical  task  compo¬ 
nents.  Critical  mission  states  in  which  accurate 
or  rapid  task  performance  is  essential  can  be 
revealed  by  an  analysis  of  the  mathematical 
structure  of  the  SYSPM  function.  Finally, 
SYSPM's  permit  rapid  assessment  of  perfor¬ 
mance  and  provide  a  basis  for  KOR  (knowledge 
of  results)  feedback  for  training  enhancement. 

A  Rule  for  Calculating  the  SYSPM 

From  the  theory  of  performance  measure¬ 
ment  there  follows  as  a  major  result  a  rule  for 
calculating  the  SYSPM  over  an  interval  of  time. 
Given  this  rule,  performance  can  be  assessed 
over  any  length  interval  of  time  desired  -  over 
the  total  mission  itself,  for  example,  or  over 
any  portion  of  the  mission  -  by  summing  over 
successive  (preferably  short)  time  intervals . 

The  rule  for  calculating  the  SYSPM  over  the 
ith  interval  is  the  following: 

SYSPM.  =  AR.  +  Ru.  forAR.  +  Ru.  N  0 
i  v  i  i  1/ 

SYSPM.  =  0  for  AR.  +  Ru.^  0  J 

Where  AFT  (R.  +  1  -  R.); 

R.  =  the  resources  required  to  complete 
the  mission  from  the  beginning  of 
the  Ith  interval,  given  that  reference 
performance  is  exhibited  throughout 
the  remainder  of  the  mission,  and 

Ru.  =  the  resources  that  are  in  fact  used 
during  the  i  interval.  Note  that 
by  convention  Ru^  )>  0,  l.e. ,  the 
resources  used  are  always  specified 
by  a  positive  number. 

The  sum  over  all  intervals  of  the  SYSPM  is 
the  value  of  the  SYSPM  over  the  entire  mission; 
or: 

SVSPM  =  ^  ^  SYSPM.  =  SUMPM 
all  i 

Similarly,  the  value  of  the  SYSPM  over  any 
specified  portion  of  the  mission  is  the  sum  of  the 
SYSPMj  over  the  intervals  constituting  that 
portion. 

In  practical  situations  it  is  not  generally 
convenient  to  think  in  terms  of  equal-length  in¬ 
tervals  defined  by  a  running  clock.  Instead,  the 
actual  mission  is  first  broken  down  into  its 


separate  state  transitions,  which  may  be  further 
broken  down  into  their  constituent  tasks.  The 
intervals  then  correspond  to  the  state  transitions, 
or,  in  a  more  fine-grained  approach,  to  the  time 
actually  taken  to  complete  each  task.  As  the 
mission  progresses,  a  sequence  of  Ini  ervals  Is 
generated  as  each  transition  or  dlscri  :e  task  is 
accomplished. 

Discussion  of  Equation  1  .  According  to 
equatton  1 ,  the  SYSPM  over  the  Ith  Interval  is 
equal  to  the  change  in  the  resources  required  to 
complete  the  mission  assuming  that  reference 
performance  will  be  exhibited  throughout  the 
remainder  of  the  mission  plus  the  resources 
actually  used  during  the  Ith  Interval.  When  re¬ 
sources  have  been  utilized  effectively,  the  value 
of  Ruj  will  be  equal  to  the  absolute  value  of  AR., 
and  the  SYSPM(  over  the  interval  i  will  equal  1 
zero.  Thus  the  SYSPM  acts  as  a  penalty  function, 
since  its  value,  the  "penalty,"  is  zero  when 
actual  performance  equals  or  Improves  upon 
expected  performance,  but  is  positive  otherwise. 

If  the  resources  actually  used  exceed  the  absolute 
value  of  the  reduction  in  resources  required 
to  complete  the  mission  given  reference  per¬ 
formance,  resources  have  been  wasted  (in 
terms  of  the  standard  set  by  reference  perfor¬ 
mance)  and  a  penalty  is  assigned  equal  to  the 
excess  resources  used.  It  is  the  property  of 
the  SYSPM  function  over  each  interval  of  time 
(for  example,  over  each  task)  that  permits  cor¬ 
rect  performance  measurement  of  each  constituent 
task  of  each  transition  independent  of  the  level 
of  performance  of  any  tndividual(s)  on  previous 
tasks . 

For  example,  suppose  we  are  working  on 
a  project  that  should  require  30  days  to  complete. 
If  we  work  effectively  for  one  day,  the  project  at 
the  end  of  the  day  should  require  29  more  days 
to  complete.  Thus  Ri  =  (29  -  30)  =  -1 ,  Ru^  =  1 , 
and  the  SYSPM.  =  0.  If,  however,  we  do  not 
work  effectively,  and  we  require  one  and  a  half 
days  to  do  a  day's  work,  R.  =  (29  -  30)  =  -1  as 
before,  but  Ru.  •=  1.5,  and  so  the  value  of 
SYSPM.  =  0.5.  We  have  wasted  half  a  day,  and 
the  value  of  SYSPM.  penalizes  us  accordingly. 

It  is  important  to  note  here  that  the  SYSPM 
does  not  require  the  questionable  assumption  of 
equal  sensitivity  of  mission  performance  to 
each  type  of  task,  nor  does  it  require  measure¬ 
ment  of  deviation  from  a  fixed  scenario  ^ath. 

The  SYSPM  will  properly  assign  each  penalty  as 
it  occurs  without  applying  to  a  given  task  a 
penalty  due  to  poor  performance  on  previous 
tasks . 

With  regard  to  equation  1  we  now  note  further 
that  the  measurement  of  the  effect  on  mission  per¬ 
formance  of  the  performance  of  a  specific  task 
does  require  a  statement  of  how  the  remainder 
of  the  mission  ought  to  be  accomplished.  To 
satisfy  this  requirement,  we  have  developed  the 
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concept  of  reference-task  performance.  When 
reference-task  performance  continues  throughout 
the  remainder  of  the  mission,  we  are  required  to 
state  exactly  what  amount  of  reference  resources 
(such  as  time,  funds,  fljel,  ammunition,  and  so 
forth)  will  be  used  in  each  one  of  the  mission’s 
remaining  stages .  This  formulation  of  the 
quantity  R.  is  the  major  source  of  difficulty  in 
calculating  the  SVSPM. 

The  Specification  of  Reference  Performance 

Essential  to  the  validity  of  u3lng  equation  1 
for  calculating  the  SVSPM  over  the  itb  interval 
is  the  assumption  that  reference-task  performance 
will  be  exhibited  throughout  the  remainder  of  the 
particular  mission.  In  this  respect,  reference- 
task  performance  may  be  regarded  as  a  prediction 
of  activity  yet  to  be  demonstrated,  as  a  prediction 
of  the  capabilities  of  the  present  system. 

Originally,  the  flight  control  systems  to 
which  the  SYSPM  was  applied  were  limited  in 
their  performance  entirely  by  the  hardware  in 
use,  rather  than  by  human  factors.  Because 
these  limitations  were  fully  represented  by  known 
equations,  it  was  possible,  given  the  SUMPM,  to 
define  exactly  the  optimal  performance  and  tc 
use  it  as  the  standard  for  reference  performance . 
Where  the  SUMPM  was  a  function  of  response 
time,  for  example,  reference  performance  for 
the  system  was  regarded  as  the  time-optimal 
performance;  where  the  SUMPM  was  a  function 
of  fuel  use,  reference  performance  occurred 
when  the  least  fuel  was  used .  The  reader  is 
referred  to  Connelly,  et  al.(1977)  for  a  detailed 
presentation  of  the  computation  of  the  SYSPM  for 
flight  control  systems . 

In  a  computerized  system  i.ivolving  human 
operators,  where  the  factors  limiting  performance 
are  as  a  rule  not  embodied  in  any  known  equations 
governing  the  system,  optimal  performance 
simply  cannot  be  determined  analytically. 
Nevertheless,  the  SYSPM  may  be  determined  by 
recognizing  that  the  factors  limiting  performance 
are  embedded  in  actual  demonstrations  of  the 
computerized  system,  i.e.,  in  real-life  demonstra¬ 
tions  given  by  human  operators  using  the 
computerized  equipment.  From  such  demonstra¬ 
tions,  the  limited  performance  can  be  documented 
and  the  reference  performance  can  be  empirically 
established. 

To  provide  a  framework  for  the  collection  of 
the  necessary  data,  and  to  facilitate  its  collection 
from  a  variety  of  computerized  systems  so  that 
it  may  be  applied  to  the  specification  of  reference- 
task  performance,  a  set  of  parameterized  generic 
tasks  has  been  developed.  Ultimately,  it  is  hoped 
to  have  tables  for  each  of  these,  so  that  the 
execution  time  and  error  rates  for  each  task  type 
(entering  data  via  keyboard,  selecting  a  data 
format,  etc.)  and  each  parameter  of  a  task  (the 
number  of  keys  on  a  keyboard,  for  example)  are 
readily  available.  Thus  far,  from  the  literature 


and  from  a  field  study  to  be  described  subsequently 
(Connelly,  et  al.,  in  press),  only  some 
of  the  generic  tasks  have  been  well  defined. 
More  work  in  this  area  is  still  required. 


REVIEW  AND  DISCUSSION 

This  study  utilized  a  model  of  an  Army 
tactical  data  processing  system  (TACFIRE)  in 
which  the  performance  of  each  team  member  on 
each  task  of  a  specific  mission  (the  High  Burst/ 
Mean  Point  of  Impact  Mtssion)  was  represented. 
In  order  to  Implement  the  SYSPM,  one  must 
assess  for  each  task  as  it  is  accomplished  its 
effect  on  overall  mission  performance  in  ternv» 
of  deviations  from  expected  task  performance, 
which,  in  turn,  is  based  entirely  upon  reference- 
task  performance  and  the  transition  probabilities. 
As  a  result,  reliable  and  (preferably)  extensive 
data  are  required  to  specify  the  reference-task 
values  and  the  transition  probabilities.  The 
results  of  the  study  are  too  voluminous  to 
discuss  at  this  time.  For  ftjrther  discussion 
see  Connelly,  et  al.  (in  press). 


CONCLUSIONS 

By  presenting  in  full  detail  both  the  develop¬ 
ment  of  the  SYSPM  as  an  evaluative  tool  as 
well  as  its  application  to  a  problem  of  continuing 
interest  to  Army  team  training,  this  report 
has  sought  to  draw  attention  to  several  broad 
and,  at  the  same  time,  practical  conclusions. 
Namely  that: 

1  .  Data  collection  in  the  field  can  be 

satisfactorily  accomplished  by  inter¬ 
viewing  system  experts. 

2 .  The  utility  of  the  generic  method  of 
classifying  tasks  now  having  been 
demonstrated,  the  method  can  be 
extended  to  include  virtually  all  tasks 
involved  in  the  operation  of  any  team/ 
computerized  system. 

3 .  The  System  Performance  Measure, 
SYSPM,  can  be  applied  successfully  to 
any  tactical  data  processing  system. 

4.  Computer  scoring  of  the  SYSPM  for 
most  tasks  may  be  easily  accomplished 
with  only  moderate  compute-time  and 
memory  demands.  With  the  storage  of 
the  expected  resources  required  for  each 
task  at  any  point  in  a  misston  complete, 
the  only  work  remaining  to  be  done  is 
the  rather  simple,  simultaneous  evalua¬ 
tion  of  equation  1  .  Of  course  certain 
tasks ,  particularly  those  interactive 
tasks  not  involving  a  computer,  may 
always  have  to  be  scored  manually  in 
some  check-list  fashion. 
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5.  The  information  automatically  provtded  a 
computer-scored  verston  of  the  SYSPM, 
particularly  with  respect  to  the  speed/ 
accuracy  trade-off,  is  exactly  the  type 
required  by  an  automatic  team  training 
system  for  effectively  instantaneous 
conditioning  feedback. 
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ABSTRACT 


Since  World  War  II,  the  U.S.  Army  has  considerably  expanded  its  use  of  the  helicopter  In 
a  variety  of  military  functions.  As  new  missions  were  defined,  new  tactics,  extended  perform¬ 
ance  requirements  and  increased  number  of  subsystems  have  imposed  extreme  demands  on  the  pilot. 
Ground-based  flight  simulation  is  the  only  safe  practical  way  to  investigate  the  tradeoffs  be¬ 
tween  a  better-trained  pilot  and  a  more  complex  aircraft.  In  1975,  a  joint  U.S.  Army  and  NASA 
study  was  performed  to  establish  the  future  needs  for  the  simulation  of  rotary-wing  aircraft. 
As  a  result,  a  program  was  initiated  to  develop  a  facility  that  could  be  used  by  government 
and  industry  in  research  and  development.  That  facility  is  being  developed  jointly  by  the  U.S. 
Army  and  NASA  at  the  Ames  Research  Center. 

In  1978  the  Franklin  Research  Center  completed  the  development  of  the  concept  for  the 
motion  generator  to  satisfy  the  requirements  of  the  new  simulation  facility.  In  1979  they  be¬ 
gan  the  design  of  the  unit  which  is  to  be  installed  at  the  Ames  Research  Center  in  1982.  The 
Rotorcraft  Simulator  Motion  Generator  (RSMG)  is  a  new  four-degree-of-freedom  system  to  replace 
the  synergistic  motion  system  presently  mounted  on  the  Vertical  Motion  Simulator  at  Ames.  Its 
extended  capabilities  will  satisfy  the  requirements  for  research  involving  both  fixed-wing  and 
rotary-wing  aircraft.  In  this  way  the  Army/NASA  goals  for  an  advanced  facility  for  rotorcraft 
simulation  are  to  be  satisfied  most  efficiently. 


INTRODUCTION 

Although  the  U.S.  Army  accepted  delivery  of  its 
first  helicopter  40  years  ago,  it  was  not  until 
af+r'  the  Korean  War  that  the  necessary  doctrine  and 
experience  were  available  with  which  the  development 
of  a  military  helicopter  could  begin  in  earnest. 

The  greatest  impulse  to  progress  in  helicopter  de¬ 
velopment  resulted  from  the  requirements  and  experi¬ 
ences  in  the  Korean,  Viet  Nam  and  Middle  East  wars. 

In  the  three  decades  since  the  end  of  World 
War  II,  the  U.S.  Army  has  considerably  expanded  its 
use  of  the  helicopter.  Originally,  the  helicopter 
was  thought  of  as  being  a  reconnaissance,  evacuation 
and  general-purpose  aircraft  that  was  capable  of 
performing  missions  similar  to  those  that  had  been 
performed  by  the  light,  fixed-wing  aircraft.  As 
the  potential  of  this  vehicle  began  to  be  appreci¬ 
ated,  its  use  added  another  dimension  to  the  battle¬ 
field  by  enhancing  the  Army's  ability  to  conduct  the 
land  combat  functions  of  mobility,  intelligence, 
firepower,  combat  service  support,  and  command,  con¬ 
trol  and  communication.  Helicopters  are  now  recog¬ 
nized  by  the  U.S.  Army  as  important  replacements  for 
traditional  ground  vehicles  in  the  performance  of 
certain  missions  that  are  beyond  the  capability  of 
fixed-wing  aircraft.  As  the  helicopter  has  acquired 
these  new  missions,  it  has  also  acquired  new  tactics, 
new  performance  requirements,  and  a  tremendous  in-  - 
crease  in  the  number  of  subsystems,  most  of  which  re¬ 
quire  some  degree  of  management  or  control  by  the 
pilot. 

Training  alone  may  no  longer  enable  the  pilot  to 
cope  with  the  situation.  It  is  possible  that  regard¬ 
less  of  the  extent  of  training,  we  are  approaching 
the  limit  of  the  human  pilot's  capability.  Of  course 
the  helicopter  could  be  made  easy  to  fly  or  even  to 
fly  itself  in  these  new  missions,  but  such  benefits 
are  costly.  Automation  can  significantly  increase 


cost  and  complexity,  and  adversely  affect  reliability 
and  maintainability.  To  be  cost  effective,  the 
military  helicopter  must  make  full  use  of  its  pilot 
and  his  capabilities.  However,  he  must  not  be  over¬ 
loaded  to  the  extent  that  his  mission  performance  is 
degraded  or  his  margins  for  error  are  decreased  un¬ 
til  there  is  an  increased  susceptibility  to  accidents. 
Ground-based  flight  simulation  is  the  only  safe  and 
practical  way  to  investigate  the  trade-offs  system¬ 
atically  before  hardware  is  developed. 

Over  the  last  20  years  or  so,  ground-based 
flight  simulation  has  become  a  recognized  and  widely 
accepted  training  tool.  In  the  fixed-wing  aircraft 
industry,  the  cost  effectiveness  of  ground-based 
flight  simulation  in  research  and  development  has 
also  been  demonstrated  (1).  Flight  simulators  have 
been  used  to  a  far  lesser  extent  by  the  rotary-wing 
industry.  In  1975,  a  joint  U.S.  Army  anc  NASA  study 
was  performed  to  review  the  functions,  status  and 
future  needs  for  ground-based  flight  simulation  of 
rotary-wing  aircraft.  In  the  course  of  this  review, 
the  deficiencies  in  current  simulation  capability 
relative  to  rotary-wing  aircraft  requirements  were 
identified.  As  a  result  of  that  review  (2),  a  pro¬ 
gram  was  initiated  to  develop  a  high-fidelity  rotor¬ 
craft  simulation  capability  that  could  be  exploited 
by  both  government  and  industry  in  research  and  de¬ 
velopment.  The  simulation  capability  is  being  de¬ 
veloped  jointly  by  the  U.S.  Army  and  NASA  at  Ames 
Research  Center. 

USES  OF  A  ROTORCRAFT  SIMULATOR 
IN  RESEARCH  AND  DEVELOPMENT 

The  1975  Army/NASA  study  concluded  that  the  needs 
for  a  helicopter  R&D  simulator  fell  into  the  follow¬ 
ing  two  categories: 

1.  In  support  of  basic  technology.  This  work 
consists  of  generic  studies  of  stability 
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and  control,  handling  qualities,  controls 
and  displays,  and  other  aspects  of  the 
man-machine  Interface. 

2.  In  support  of  the  development  of  new  avia¬ 
tion  systems  or  Improvements  to  fielded 
systems.  These  efforts  start  early  In  an 
aircraft  acquisition  cycle  by  assisting 
the  user  and  the  developer  In  performing 
design  studies,  system  Integration  evalua¬ 
tions  and  trade-offs. 

The  first  of  these  uses  permits  us  to  address 
the  fact  that  current  helicopter  flying  qualities 
specifications  are  based  on  an  obsolete  design 
standard.  For  our  newest  helicopters,  we  have  had 
to  devise  poorly  substantiated  criteria  for  new 
missions  and  tasks.  Therefore,  In  our  current  R&D 
program  we  are  pursuing  the  development  of  a  tech¬ 
nological  data  base  in  rotorcraft  handling  qualities 
that  should  enable  us,  for  the  first  time,  to  gen¬ 
erate  the  criteria  and  the  specifications  on  flying 
qualities  for  rotary-wing  aircraft  designed  to  per¬ 
form  military  missions  (Figure  1).  Ultimately,  the 
intent  is  to  provide  the  designer  with  the  matrix 
of  information  he  needs  to  relate  effectiveness  to 
life-cycle  costs. 


NOE CRUISE 


FLIGHT  CONTROLS 

•  MULTI-MODE  SCA5 

•  INTEGRATED 
PILOT  CONTROLS 
AND  DISPLAYS 


ORDNANCE  DELIVERY 


Figure  1.  Helicopter  Handling  Qualities  Research 


The  development  of  a  handling-qualities 
specification  for  use  by  helicopter  manufacturers 
in  the  design  phases  would  benefit  both  the  indus¬ 
try  and  the  government.  Experience  has  shown  that 
the  use  of  the  current  handling-qualities  specifi¬ 
cation  (MIL-H-8501A)  has  failed  to  provide  more 
than  basic  guidance  to  industry  and  attempts  to 
meet  the  requirements  of  that  specification  have, 
in  many  instances,  resulted  in  undesirable  flying 
qualities  (3).  Individual  specifications  were  de¬ 
veloped  for  the  Utility  Tactical  Transport  Aircraft 
System  (UTTAS)  and  the  Advanced  Attack  Helicopter 
(AAH)  In  an  effort  to  eliminate  this  deficiency; 
but  both  helicopters,  although  judged  to  have 
superior  flying  qualities,  also  failed  to  meet  cer¬ 
tain  requirements  of  their  specifications  (4). 

From  an  aeromechanics  point  of  view,  our  most  modern 
U.S.  Army  aircraft,  the  UTTAS  and  the  AAH,  are  based 
on  technology  that  is  10  to  20  years  old.  These 
aircraft,  like  their  predecessors,  will  impose  work¬ 
loads  on  their  aircrews  during  typical  Army  missions 
that  will  constrain  the  pilot  from  exploiting  to  the 
maximum  the  full  capabilities  of  his  aircraft,  es¬ 
pecially  at  night  or  under  adverse  weather  condi¬ 
tions. 


Rotor  systems  and  their  associated  controls 
offer  the  most  direct  method  of  Improving  flying 
qualities  and  reducing  pilot  workload  In  the  missions 
and  tasks  typically  assigned  to  Army  helicopters. 

Chen  and  Talbot  (5)  investigated  four  major  rotor 
system  design  parameters  to  assess  the  handling 
qualities  for  44  configurations  of  main-rotor  systems 
that  cover  teetering,  articulated,  and  hingeless 
families  of  rotor  systems  with  a  wide  range  of  blade 
Inertia.  They  concluded  that  within  each  family 
of  rotor  systems,  satisfactory  handling  qualities 
could  be  obtained  with  the  appropriate  combination 
of  rotor  parameters.  However,  no  single  rotor 
system  was  uniformly  superior  In  all  aspects  of 
handling  qualities  during  typical  operations. 
Additional  experiments  such  as  these  are  required  to 
optimize  the  handling  qualities  for  specific  mis¬ 
sions. 


Figure  2.  Systems  Development  Cycle 


The  second  use  of  R&D  flight  simulators,  during 
the  development  of  new  aviation  systems  for  improve¬ 
ments  to  fielded  systems,  follows  the  entire  life 
cycle  of  system  development  (Figure  2).  During  the 
program  initiation  phase,  the  simulator  can  be  used 
to  evaluate  new  aviation  concepts  or  tactics  that 
have  been  developed  by  the  U.S.  Army  Training  and 
Doctrine  Command  (TRADOC)  to  meet  a  specific  threat. 
The  R&D  simulator  also  provides  an  ideal  environment 
for  evaluating  the  threat  from  both  ground  weapons 
and  enemy  helicopters.  The  probability  of  air-to- 
air  combat  between  helicopters  on  the  future  battle¬ 
field  is  extremely  high.  Success  in  these  engage¬ 
ments  may  depend  on  exploitation  of  weakness  in  the 
threat  helicopter's  handling  qualities  or  in  the 
optimization  of  our  own  flight  maneuvers.  It  may  be 
in  this  approach  to  establishing  requirements  that 
ground-based  simulators  will  play  their  most  effec¬ 
tive  role  in  minimizing  the  life-cycle  cost  of  our 
future  aircraft.  Such  evaluations  can  help  answer 
the  questions  and  support  the  rationale  leading  to 
a  Mission  Element  Needs  Statement  (MENS).  After 
the  MENS  is  approved,  the  R&D  simulator  can  be  used 
In  the  demonstration  and  validation  phase  for  evalu¬ 
ating  the  flying  qualities  of  competing  designs  as 
wtll  as  for  easing  future  systems  integration  ef¬ 
forts. 

Manned  simulation  also  plays  an  important  role 
in  establishing  hardware  configuration  during  the 
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development  of  the  helicopter.  During  the  evalua¬ 
tion  phase  of  a  baseline  design,  test  pilots  and 
operational  pilots  are  provided  the  opportunity, 
through  manned  simulation,  to  evaluate  the  baseline 
and  mission  scenarios  with  full  operational  freedom. 
This  Is  the  last  point  In  time  when  changes  to  the 
baseline  design  can  be  made  without  extremely  costly 
hardware  retrofit.  Also,  actual  prototype  flight 
hardware  can  be  Incorporated  Into  the  flight  simula¬ 
tor.  Although  standard  bench  integration  test  will 
verify  electrical  and,  In  some  cases,  software  com¬ 
patibility,  only  a  dynamic  simulation  can  completely 
exercise  the  equipment.  Even  more  important,  all 
aspects  of  the  software  can  be  tested  in  a  mission 
envlroment  well  before  the  aircraft  flies. 


Figure  3.  Aircraft  Accident  Investigation 


Finally,  the  R&D  simulator  can  be  used  to  in¬ 
vestigate  unusual  accidents  (Figure  3),  the  under¬ 
standing  of  which  defines  normal  investigative 
techniques.  One  such  investigation  has  already 
been  accomplished  at  Ames  Research  Center.  In 
March  1976,  a  Bell  Helicopter  Textron  Model  214 
helicopter  crashed  during  hardover-control -signal 
testing  of  its  Automatic  Flight  Control  System 
(AFCS).  The  subsequent  accii  investigation  did 
not  conclusively  establish  ’.he  cause  of  the  accident 
but  did  indicate  that  it  was  not  caused  by  a  mech¬ 
anical,  electrical,  or  hydraulic  failure.  It  was 
decided  to  continue  the  investigation  using  the 
si x-degree-of- freedom  Flight  Simulator  for  Advanced 
Aircraft  (FSAA)  at  Ames  Research  Center.  The  re¬ 
sults  proved  that  removing  the  hardover-control - 
signal  at  the  same  time  the  pilot  was  taking  correc¬ 
tive  action  causes  large  spikes  in  blade  flapping 
and  was  the  probable  cause  of  the  accident.  The 
procedure  for  hardover-control -signal  testing  was 
subsequently  modified  and  similar  accidents  have 
not  occurred. 

In  sumnary,  flight  -  Nation  ir  .«  important 
tool  In  helicopter  research  and  development,  both 
for  technology-base  development  and  for  aircraft  de¬ 
velopment  programs.  There  is  no  question  that 
ground-based  simulation  has  been  and  will  continue 
to  be  an  invaluable  tool.  The  flight  simulator  is 
to  the  flight  dynamicist  what  the  wind  tu,  is  to 

the  aerodynamici st.  The  emphasis  on  the  ol  of 

development  costs  and  operational  trainir'„  .osts 
suggests  that  flight  simulators  will  play  an  in¬ 
creasingly  important  role  in  future  research  and  de¬ 
velopment  of  Army  rotary-wing  aircraft. 


NOE  ®  CONTOUR  ®  LOW  LEVEL  © 


Figure  4.  Terrain  Flying  Regimes 


REQUIREMENTS  OF  A  RESEARCH  AND  DEVELOPMENT 
ROTORCRAFT  SIMULATOR 

The  modern  battlefield  has  become  a  highly 
lethal  place  for  both  fixed-  and  rotary-wing  air¬ 
craft.  The  formidable  array  of  weapons  that  can  be 
used  against  aircraft  has  forced  pilots  to  abandon 
their  normal  operating  altitudes  in  the  vicinity  of 
a  battlefield.  The  only  air  space  that  can  be  con¬ 
sidered  relatively  safe  is  below  100  feet  and  then 
only  if  a  sufficient  amount  of  ground  cover  is  avail¬ 
able.  The  helicopter  is  naturally  a  ground  contact 
machine  par  excellence  and  its  mission  use  in  Army 
aviation  is  more  characteristic  of  a  flying  jeep  or 
tank  than  of  an  airplane.  Helicopters  fly  low  and 
slow  and,  especially  during  military  missions,  are 
close  to  the  ground  during  most  of  their  flying 
time.  The  term  nap-of-the-earth  (NOE)  (Figure  4) 
has  been  coined  by  the  helicopter  community  to  des¬ 
cribe  operations  in  which  helicopters  fly  only  a 
few  feet  above  the  ground  and  fly  around  obstacles 
rather  than  over  them.  The  environment  for  the 
pilots  flying  these  missions  is  rich  in  detail  — 
trees,  bushes,  hills  and  valleys.  Although  these 
terrain  features  offer  protection  from  the  enemy, 
they  can  be  lethal  to  an  unwary  pilot.  In  addition, 
visibility  factors  associated  with  weather  and 
darkness,  and  atmospheric  characteristics  of  wind, 
turbulence  and  ground  effect  are  all  elements  of 
the  environment  that  may  significantly  affect  the 
helicopter  pilot's  tasks.  The  helicopter  crew  must 
maneuver  around  and  between  obstacles,  and  navigate, 
communicate  and  proceed  with  the  mission  while 
maintaining  awareness  of  threat  weapons. 

Current  simulation  capabilities  cannot  meet 
the  requirements  of  rotary-wing  aircraft  when  one 
considers  all  the  aspects,  including  mission,  task, 
aircraft  characteristics,  environmental  conditions, 
instrumentation  and  displays,  performance  and  work¬ 
load.  Many  of  these  aspects  impose  requirements 
quite  different  from  those  met  by  even  the  most 
sophisticated  fixed-wing  simulators.  The  most  ad¬ 
vanced  ground-based  simulators  in  the  world  are 
available  to  the  U.S.  Army’s  Aeromechanics  Laboratory 
(through  agreements  with  Ames  Research  Center),  but 
even  these  are  not  adequate  to  meet  the  Army's  need 
to  simulate  nap-of-the-earth  flight  operations.  The 
visual  display  is  required  to  represent  much  more 
detail  in  the  terrain  and  vegetation.  Low  flight 
speeds  and  high  maneuverability  allow  rapid  changes 
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of  fllghtpath  to  be  achieved  so  that  the  field  of 
view  required  for  the  helicopter  pilot  to  see  where 
he  Is  going  Is  wider  than  that  of  a  fixed-wing  air¬ 
craft.  This  paper,  however,  concerns  Itself  with 
the  requirement  for  motion.  Deel  md  Rue  discussed 
visual  concepts  (6)  during  the  last  conference. 


Motion  (Platform)  Requirements 

There  Is  no  obvious  and  accepted  measure  of 
motion  cue  requirements.  It  Is  generally  agreed 
that  motion  simulation  Is  required:  (1)  when 
expected  motions  are  above  human  sensory  or 
Indifference  thresholds;  (2)  when  expected 
motions  are  within  the  sensory  frequency  range, 
that  Is,  above  0.2-0. 5  rad/sec;  (3)  If  full 
pilot  performance  (e.g.  tracking)  Is  desired; 
and  (4)  when  a  degree  of  face  validity  or  real¬ 
ism  is  required  to  gain  pilot  acceptance  of  the 
total  simulation. 

An  example  of  relating  simulator  motion  sys¬ 
tem  capabilities  to  the  maneuver  envelope  of  an 
aircraft  Is  presented  In  a  paper  by  Key  et  al 
(7),  which  Includes  a  description  of  the  develop¬ 
ment  of  the  requirements  for  a  motion  system  to 
be  used  In  a  helicopter  flight  simulator. 


Parana car 

Position, 

Velocity, 

Acceleration, 

Axis 

rad. 

rad/aee. 

rad/aee2, 

■ 

a/aec 

n/aee2 

Yaw 

40.4 

+0.6 

41.0 

Pitch 

♦0.3 

+0.5 

41.0 

Roll 

40.  .v 

+0.5 

41.0 

Surge 

41.3 

41.3 

♦3.0 

Sway 

43.0 

♦2.6 

^3.0 

H«V* 

47,  -14 

♦8,  -11 

414,  -12 

Table  1.  Motion  (Platform)  Requirements  for 
Critical  Terrain  Flight  Maneuvers 
(from  Reference  7) 

The  criteria  that  were  adopted  for  these  re¬ 
quirements  were  based  on  the  opinions  of  ex¬ 
perienced  researchers,  which  in  turn  were  supported 
oy  limited  test  data.  Flight  maneuvers  resulting 
from  fixed-base  simulations  of  NOE  flight  opera¬ 
tions  were  analyzed  to  define  the  platform  excur¬ 
sion  requirements.  These  time  histories  were 
played  (off-line)  through  a  drive  logic  represent¬ 
ing  that  of  an  advanced  si x-degree-of- freedom 
simulator,  with  the  fidelity  boundaries  and  selec¬ 
ted  operating  points  for  each  axis.  The  results 
of  the  analysis,  in  terms  of  the  maximum  excursion, 
velocity,  and  acceleration  of  each  axis,  are  pre¬ 
sented  in  Table  1.  The  requirement  is  that  all 
axes  produce  these  quantities  simultaneously;  this 
requirement  is  amplified  by  the  data  of  Table  2, 
where  the  position  of  each  axis  at  the  instant 
that  one  axis  reached  a  naximum  is  presented.  The 
data  are  from  a  typical  maneuver  case.  The  sig¬ 
nificance  of  the  data  is  then  when  one  axis  is  at 
a  maximum,  some  of  the  others  are  at  large  values 
also.  A  nonlinear  drive  logic  is  needed  to  vary 
the  gains  and  washout  frequencies  with  amplitude 
of  motion  in  order  to  obtain  as  much  fidelity  as 
possible  for  lower  amplitude  tasks. 
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Table  2.  Examples  of  Simultaneous 

Excursions  (from  Reference  7) 


RSIS  PROJECT  PLAN 

Under  joint  agreement,  Ames  Research  Center 
and  the  U.S.  Army  Research  and  Technology  Labora¬ 
tories,  Aviation  Research  and  Development  Coimand 
(AVRADCOM),  have  agreed  to  acquire  the  Rotorcraft 
Systems  Integration  Simulator  (RSIS)  to  be  in¬ 
stalled  at  Ames  Research  Center.  The  program  Is 
now  In  Its  final  phase.  The  definition  phase 
started  with  an  Army/NASA  study  In  1975  which  led 
to  additional  studies  to  address  the  Issues  raised 
by  the  special  requirements  of  rotorcraft  simula¬ 
tion.  A  feasibility  study  of  a  wide-angle  visual 
simulation  system,  completed  by  Northrop  In  1977, 
showed  that  a  wide  fleld-of-vlew  display  (120° 
horizontally  by  60°  vertically)  was  feasible. 
Analyses  of  fixed-base  and  motion-base  simulations 
of  NOE  flight  operations  have  defined  the  cab  ex¬ 
cursions  required  for  high-fidelity  simulation 
motion.  It  was  determined  that  the  Vertical 
Motion  Simulator  (VMS)  at  Ames  Research  Center 
could  be  modified  ar.d  used  as  the  motion  base  of 
the  RSIS  (Figure  5).  Independent  design  studies 
to  assess  the  possible  modification  to  the  VMS  were 
performed  by  Franklin  Research  Center  and  Northrop 
Corporation  in  1978.  Specifications  were  developed 
from  those  two  studies,  a  competitive  request  for 
proposal  was  Issued  to  industry  and  the  contract 
was  awarded  to  Franklin  Research  Center  In  1979. 

The  modification,  known  as  the  Rotorcraft  Simula¬ 
tor  Motion  Generator  (RSMG),  will  be  delivered  in 
late  1982.  The  remainder  of  this  paper  will  dis¬ 
cuss  the  design  and  the  fabrication  of  the  RSMG 
by  the  Franklin  Research  Center  (FRC). 


Figure  5.  The  Vertical  Motion  Simulator 
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DESIGN  STUDIES  FOR  THE 
ROTORCRAFT  SIMULATOR  MOTION  GENERATOR  (RSMG) 

Motion  Generator  Specification 

As  a  result  of  the  analyses  performed  by  the 
U.S.  Army  Aeromechanics  Laboratory  and  the  National 
Aeronautics  and  Space  Administration  at  the  Ames 
Research  Center,  specifications  were  developed  for 
the  Rotorcraft  Simulator  Motion  Generator  (RSMG) 
as  shown  In  Table  3.  The  severe  limitation  on 


1 .  Pyr  forntiK'v 

Simultaneous 

Node _ _ _ _ _ 

Displacement  _ 

.  Velucl ties 

_  A^'«dL®r. jiti  on_ 
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♦  l.?2  m 

♦  1.22  m/sec 
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(+  4  ft.) 

(♦  4  ft. /see) 
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+  0,314  rad 
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+  0,314  rad 
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(+  18°) 

<+  k 0°/«vc) 
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Yaw  (♦) 

+  0.418  rad 

+0.8  rad /see 

+  2.0  rad/sec^ 

2.  Payload 

<±  24°) 

(♦  46°/sec) 

(.+  m°/,ec2) 

«  Configuration 

20.5 

ft.  dla.  sphere  section 

•  Gross  Weight 

8000 

-  12.000  lbs. 

•  Moments  of  Inertia  3000 

).  Frequency  Response 

-  20,000  lbs.  ft.  f 

<vc  1 

•  Second  order  system  natural  freq.  3  Hz  and  damping  factor  0.7 

•  Tolerances ,  +  2  db  and  +_  20  degrees 


4.  Weight  Limitation 

Total  weight  of  4D0F  system  <  16.000  lbs. 

Table  3.  Specifications  for  the  Rotorcraft 
Simulator  Motion  Generator  (RSMG) 


the  weight  of  the  RSMG  was  Imposed  so  the  perform¬ 
ance  of  the  Vertical  Motion  Simulator  (VMS)  would 
not  be  degraded  from  Its  original  performance 
goals.  In  addition  there  were  severe  constraints 
on  the  operating  envelope  of  the  RSMG  due  to  the 
Internal  dimensions  of  the  existing  VMS  building 
structure.  Most  critical,  of  course.  Is  in  the 
direction  of  longitudinal  motion  where  the  20  foot 
diameter  sphere  must  be  allowed  a  total  displace¬ 
ment  of  8  feet  within  a  building  dimension  of  31 
feet. 

Studies  of  RSMG  Candidates 

A  number  of  RSMG  configurations  were  analyzed 
In  an  effort  to  meet  all  requirements  in  the  most 
cost-effective  manner  (8).  Brief  descriptions 
follow. 

Since  the  synergistic  type  of  motion  system. 
Illustrated  In  Figure  6,  Is  the  most  efficient 
machine  for  generating  slx-degrees-of-freedom 
(6D0F)  motions  It  was  considered  first.  Design 
calculations  showed  that  the  actuator  lengths  re¬ 
quired  to  provide  all  displacements  simultaneously 
were  unreasonably  long  (30  feet).  In  addition,  a 
failure  mode  analysis  showed  that  under  certain 
emergency  conditions,  the  platform  could  assume  an 
attitude  that  would  cause  the  20  foot  sphere  to 
strike  the  building  wall. 


Figure  6.  Synergistic  6D0F  Motion  System 


The  opposite  extreme  is  a  cascaded  system  as 
shown  In  Figure  7  where  the  sphere  is  carried  on 
three  rotational  gimbals  which  are,  in  turn, 
mounted  on  a  longitudinal  carriage.  This  config¬ 
uration  captures  the  sphere  within  the  range  of 
the  longitudinal  motion  under  all  operating  or 
emergency  conditions.  However,  the  weight  of  the 


Figure  7.  Cascaded  4D0F  Motion  System 
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cascaded  gimbals  and  carriage  structure  not  only 
severely  escalated  the  power  requirements  of  the 
drive  system,  but  also  far  exceeded  the  allowable 
limit  Imposed  on  the  total  RSMG  weight. 


Figure  8.  Synergistic  400F  Motion  System 


A  synergistic  4D0F  configuration,  shown  in 
Figure  8,  was  investigated  next.  It  has  a  hinged 
center-post  that  eliminates  lateral  motion  and 
limits  vertical  motion,  both  of  which  are  available 
from  the  basic  VMS.  Because  of  the  minimization 
of  moving  mass,  it  is  the  most  efficient  configura¬ 
tion  for  generating  the  remaining  4D0F  motions. 
However,  again  the  actuators  required  to  produce 
the  specified  displacements  simultaneously  were  un¬ 
realistically  long  (34  feet)  so  the  concept  was 
rejected. 


Recognizing  the  need  to  restrict  actuator 
lengths,  FRC  elected  to  separate  (or  decouple)  the 
translation  motion  from  the  rotational  motions  by 
using  a  carriage  moving  on  linear  ball  bearings  as 
shown  in  Figure  9.  Here  the  rotational  motions  are 
produced  with  a  synergistic  arrangement  of  three 
actuators  with  a  rigid  center  post  restraining  all 
translational  displacements.  A  detailed  design 
study  of  this  400F  configuration  revealed  that  all 
specified  performance  requirements  for  the  RSMG 
could  be  met  or  exceeded.  However,  the  overall 
height  of  the  RSMG  with  the  cockpit  and  dome  in 
place  required  an  unacceptable  compromise  in  the 
available  vertical  displacement  of  the  VMS. 


YAW 


Figure  10.  Final  RSMG  Configuration 

The  overall  height  was  reduced  by  decoupling 
the  yaw  motion  from  pitch  and  roll  as  shown  in 
Figure  10.  Here  the  center-post  is  mounted  on  a 
large  diameter  ball-bearing  carried  on  the  longi¬ 
tudinal  carriage.  The  platform  is  coupled  to  the 
center-post  with  a  simple  2D0F  universal  joint 
restricting  its  motion  to  pitch  and  roll  only. 
Since  the  actuators  can  now  be  positioned  verti¬ 
cally  and  their  length  is  relatively  short,  the 
overall  height  is  reduced  to  meet  the  original 
goals  for  vertical  motion  of  the  VMS  without 
impacting  the  ceiling  of  the  building. 


Figure  9,  Synergistic  3D0F  on  Longitudinal  Carriage  Figure  11.  RSMG  Model  (without  20  ft.  sphere) 
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The  final  configuration  of  the  RSMG  Is  shown 
In  the  model  photos  In  Figures  11  (without  the 
20  foot  sphere)  and  12  (with). 


Figure  12.  RSMG  Model  (with  20  ft.  sphere) 


•  Motion  Base 

•  Controls  and  Instrumentation 

•  Hydraulic  and  Electric  Power 

•  Dedicated  Computer 

•  Safety  Systems 

Each  of  these  subsystems  will  be  described  in 
greater  detail  In  the  sections  that  follow. 

Motion  Base 


A  sketch  of  the  configuration  of  the  motion 
base  was  shown  in  Figure  10  defining  the  four 
axes  of  displacements.  The  +  4  feet  of  longitu¬ 
dinal  motion  Is  achieved  wltfr  a  carriage  mounted 
on  32  linear  recirculating-ball  bearings  on  2 
linear  tracks.  Cross-sections  of  these  bearings 
manufactured  by  THK  Japan  are  shown  In  Figure  14. 
These  bearings  are  precision-ground  and  pre- 
loaded  to  provide  smooth  noise-free  operation 
without  lost  motion. 


AecwevUtmg  ••lit 


Figure  14.  Cross-Sections  of  Linear  Bearings 


ROTORCRAFT  SIMULATOR  MOTION 
GENERATOR  (RSMG)  DESIGN 

Description  of  the  RSMG  System 

The  total  RSMG  system  is  made  up  of  a  number 
of  subsytems  as  defined  in  the  Mock  diagram  In 
Figure  13.  The  motion  base  is,  of  course,  the 
central  element  to  which  all  other  elements  are 
dedicated.  It  is  driven  with  a  set  of  four  inde¬ 
pendent  electrohydraulic  actuators  with  feedback 
control  loops  and  instrumentation  for  stabilizing 
and  monitoring  performance.  The  control  systems 
are  serviced  by  a  hydraulic  power  supply  and 
electric  power.  The  control  systems  are  coimanded 
from  a  dedicated  minicomputer  which,  among  other 
things,  provides  the  interface  with  the  NASA  host 
computer.  The  entire  RSMG  system  is  integrate^ 
with  a  set  of  built-in  safety  systems  to  protect 
men  and  machines  in  the  event  of  any  foreseeable 
emergency  situation.  The  subsystems  defined  in 
Figure  13  are: 


Figure  13.  The  RSMG  System  Complex 


The  longitudinal  carriage  is  driven  with  a 
single  hydraulic  cylinder  centrally-located  be¬ 
tween  the  tracks  under  the  longitudinal  carriage. 
The  design  of  all  the  cylinders  on  the  RSMG  Is  a 
patented  telescoping  configuration  that  provides 
equal  effective  hydraulic  operating  areas  in 
both  directions  within  the  overall  length  of 
standard  commercial  unequal  area  cylinders. 

This  provides  for  the  symmetrical  application  of 
forces,  minimizes  the  size  of  the  servovalve  re¬ 
quired  and  optimizes  the  smoothness  of  motion. 

The  design  of  this  unique  equal -area  cylin¬ 
der  configuration  is  shown  In  Figure  15. 


Figure  15.  Typical  Equal-Area  Cylinder 


The  fixed  portion  on  the  left  is  nade  up  of 
two  concentric  hollow  tubes  slightly  shorter  than 
the  retracted  length.  The  moving  portion  is  a 
closed-end  hollow  tube  with  an  annular  flange 
that  telescopes  inside  and  between  the  fixed  tubes. 
A  seal  is  required  around  the  moving  tube  and 
piston  rings  are  required  around  its  annular  flange. 
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The  left  cylinder  port  allows  the  hydraulic 
fluid  to  enter  the  fixed  center  tube  and  Impact 
the  closed  end  of  the  moving  tube.  The  right 
cylinder  port  conducts  fluid  Into  the  annular  area 
between  the  moving  tube  and  the  fixed  outer  tube 
to  impact  on  the  annular  area  projected  by  the 
flange.  By  proper  selection  of  design  dimensions 
It  Is  clear  that  the  effective  area  of  the  closed 
end  of  the  moving  tube  can  be  made  equal  to  or 
different  from  the  effective  area  of  the  annular 
flange. 

The  +  24°  of  yaw  displacement  Is  provided  by 
rotating  a  truncated-cone  center  post  on  a  large 
diameter  crossed-roller-bearlng  located  in  the 
center  of  the  longitudinal  carriage.  It  Is 
approximately  48  inches  In  diameter  and  preloaded 
to  avoid  lost  motion.  It  Is  driven  by  a  single, 
equal-area  hydraulic  cylinder  lying  horizontal 
and  attached  between  the  longitudinal  carriage 
and  the  outer  radius  of  the  center  post. 

Pitch  and  roll  motions  of  +  18°  are  provided 
with  a  two-glmbal  system  mounted"  on  top  of  the 
rotating  center  post.  The  equal-area  drive  cylin¬ 
ders  are  connected  between  their  respective  gim¬ 
bals  and  the  base  of  the  rotating  center  post. 

From  the  above  description  it  is  clear  that, 
except  for  a  minor  geometrical  interaction  be¬ 
tween  pitch  and  roll  displacements,  the  RSMG 
motion  base  is  an  "uncoupled"  motion  system. 

That  is,  individual  motions  are  commanded  inde¬ 
pendently,  without  the  need  for  on-line  coordinate 
conversion.  This  minimizes  the  requirements  of 
the  dedicated  computer.  It  does  not  mean,  how¬ 
ever,  that  there  is  no  coupling  of  the  dynamics 
of  the  individual  motions.  Since  the  center  of 
mass  of  the  payload  does  not  correspond  with  the 
center  of  rotation  in  the  glmbal  system,  pitch 
accelerations  will  couple  reaction  forces  Into 
the  longitudinal  system  and  vice-versa. 

Controls  and  Instrumentation 

The  first  consideration  In  designing  the 
electrohydraulic  control  systems  Is  the  servo¬ 
valves.  Over  many  years  of  experience,  The 
Franklin  Institute  has  developed  a  proprietary 
servovalve  design  that  yields  electrohydraulic 
controls  that  have  a  minimum  of  unwanted  accel¬ 
erations,  comnonly  known  as  "hydraulic  bump"  or 
"acceleration  noise".  Current  experimental 
tests  under  an  Air  Force  contract  indicate 
"smoothness"  and/or  "stability"  better  than 
0.01 g  peak.  The  servovalves  for  the  RSMG 
electrohydraulic  controls  are  all  of  this  spec¬ 
ial  design.  It  involves  an  unconventional  lay¬ 
out  of  the  outlet  ports  of  the  third  stage, 
which  provides  for  more  positive  control  of  the 
outlet  flow  under  the  conditions  of  low  actuator 
velocity. 

To  insure  that  the  specified  dynamic  per¬ 
formance  will  be  achieved  In  the  operational 
system,  each  closed  loop  control  system  was 
carefully  designed,  mathematically  modelled  and 
simulated  for  computer  analyses.  The  analyses 
performed  on  each  of  the  four  closed  loop  con¬ 
trols  systems  were: 

•  Frequency  response 

•  Root  locus  plot 

•  Step  response 

•  Force  disturbance 


Since  the  RSMG  Is  a  research  simulator  Intended 
to  be  used  with  a  variety  of  cab  and  visual  dis¬ 
play  configurations,  these  analyses  were  per¬ 
formed  with  two  extreme  loads;  one  the  cab  and 
spherical  screen  described  In  the  specifications, 
tne  other  the  NASA  Interchangeable  Cab  (IC)  with 
no  external  visuals.  As  an  example  of  the  re¬ 
sults  obtained,  we  will  describe  the  design  and 
analyses  of  the  roll  control  system. 


Figure  16.  Block  Diagram  of  Roll  Control  System 


Figure  16  is  a  block  diagram  of  the  roll 
closed-loop  control  system.  It  uses  a  Trans- 
Tek  angular  position  transducer  as  the  primary 
feedback  element.  It  also  employs  a  Systron- 
Donner  angular  accelerometer  to  provide  the 
compensation  necessary  to  accommodate  the  wide 
range  of  expected  loads.  The  position  comnand 
is  an  analog  signal  from  0  to  10  volts  dc.  It 
is  compared  with  the  actual  position  feedback 
to  generate  an  error  signal  which  is  compensated 
with  acceleration  feedback  and  shaping  networks 
to  comnand  servovalve  spool  position.  The 
servovalve  has  a  Schaevltz  LVDT  spool  position 
transducer  in  a  minor  loop  to  extend  its  band¬ 
width  and  responds  accordingly  to  the  error 
command  to  deliver  flow  to  the  roll  actuator 
to  reposition  the  load  and  minimize  the  error. 


ROLL  SERVO  AT  2  LOADS  CROPS, 4> 


Figure  17.  Frequency  Response  of 
Roll  Actuator  System 
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The  frequency  response  of  the  roll  actuator 
system  Is  shown  In  Figure  17  together  with  the 
lines  defining  the  limits  set  up  In  the  speci¬ 
fication.  The  analyses  of  the  response  to  ex¬ 
ternal  force  disturbances,  such  as  coupling  of 
reaction  forces  due  to  VMS  lateral  motion. 
Indicate  completely  stable  behavior. 
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Opeuiot  Control  Panel 
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Figure  18.  Operator's  Control  Panel 


Control  Consoles 


There  are  two  electronic  consoles  required 
to  operate,  monitor  and  maintain  the  performance 
of  the  RSMG  system.  One  is  a  sloping-front  con¬ 
sole  containing  the  Operator's  control  panel, 
the  maintenance  test  panel,  the  control  system 
electronics  and  all  necessary  DC  power  supplies. 
The  Operator's  panel  is  shown  In  Figure  18  in¬ 
dicating  the  simplicity  of  starting-up  and  con¬ 
trolling  this  complex  machine  under  normal 
conditions.  Figure  19  shows  the  maintenance  test 
panel  with  the  means  to  address  each  axis  of 
motion  separately  and  perform  tests  to  Insure 
proper  performance.  In  the  upper  right  corner 
is  a  computer-aided  warning  system  that  monitors 
critical  system  parameters,  detects  trends  to¬ 
ward  allowable  limits  and  indicates  the  time  to 
go  until  the  RSMG  is  automatically  shut  down. 

This  allows  the  Operator  to  use  some  judgment 
when  he  is  in  the  middle  of  an  important  simu¬ 
lated  test  run. 


Figure  19.  Maintenance  Test  Panel 


The  second  electronic  console  is  a  tall  re¬ 
lay  rack  containing  a  POP  11/34  computer  with 
disk  unit,  computer  interface  electronics,  a 
system  monitoring  panel,  a  signal  output  panel 
and  the  safety  interlock  relays.  It  is  to  be 
located  remotely  from  the  vicinity  of  the  Opera¬ 
tor's  console  and  addressed  only  during  software 
changes  and  hardware  troubleshooting. 

Dedicated  RSMG  Computer 

The  central  element  of  the  RSMG  computer 
system  as  a  standard  POP  11/34  minicomputer  with 
a  CRT  terminal  and  dual  disk  memories.  To  couple 
this  computer  with  other  subsystems  three  special 
circuit  cards  have  been  designed  and  built. 

These  cards  are: 

•  analog  to  digital  (A/D)  converter 

•  digital  to  analog  (D/A)  converter 

•  Interface  circuits 

The  A.D  card  is  a  32  channel,  14  bit  con¬ 
verter  with  analog  multiplexers  and  Instrumenta¬ 
tion  amplifiers.  It  accepts  analog  Inputs  from 
the  NASA  host  computer,  the  safety  systems  and 
the  maintenance  test  panel,  and  converts  them  to 
digital  signals. 

The  D/A  card  is  a  16  bit  converter  with 
chopper-stabilized  operational  amplifiers.  The 
interface  card  contains  circuits  for  accepting 
NASA  28-volt  logic  and  operating  lamps.  It  also 
operates  the  "time  to  go"  display. 

The  RSMG  computer  system  performs  three 
major  functions: 

o  signal  extrapolation 

•  safety  monitoring 

a  maintenance 

The  NASA  host  computer  provides  Incremental 
analog  position  and  velocity  command  signals 
that  are  updated  only  every  20  milliseconds. 

The  RSMG  computer  uses  the  velocity  signal  to 
extrapolate  intermediate  points  every  2  milli¬ 
seconds,  thereby  making  the  step  change  in  posi¬ 
tion  undetectable. 

The  RSMG  computer  is  one  link  in  a  redun¬ 
dant  safety  system.  It  not  only  conducts  an 
orderly  start  up  or  shut  down,  it  also  monitors 
all  interlocks,  limits  incoming  signals  and 
performs  self-checking  routines.  The  computer 
supplements  the  maintanance  control  panel  in 
aiding  set-up,  troubleshooting  and  demonstra¬ 
tion  of  the  RSMG  system.  It  also  contains  the 
software  to  aid  in  setting  up  and  checking  out 
the  computer  system  itself. 

Safety  Systems 

The  RSMG  is  designed  with  safety  as  a  pri¬ 
mary  goal.  Emergency  systems  are  designed  to 
handle  three  levels  of  potenti ally-dangerous 
situations: 

•  excessive  commands 

•  subsystem  malfunctions 

•  total  loss  of  power 
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Level  1  systems  are  built  Into  the  electronic 
circuits.  At  the  input  to  each  electrohydraullc 
actuator  control  loop  there  are  limiting  circuits 
to  prevent  excessive  commands  for  position, 
velocity  and  acceleration.  There  are  also 
"smart"  circuits  for  limiting  the  Impact  of  run¬ 
ning  into  the  displacement  limits  of  the  actua¬ 
tors.  They  continuously  monitor  position  aril 
velocity  to  determine  the  point  where  the  actua¬ 
tor  must  start  decelerating  at  a  safe  level  as 
it  approaches  the  end  of  stroke. 

Level  2  systems  are  those  that  Incorporate 
interlocks  from  all  critical  subsystems  and 
inputs  which  automatically  shut  the  motion 
system  down  in  a  safely-controlled  manner.  Shut¬ 
down  can  be  triggered  by  a  variety  of  interlocked 
inputs  such  as: 

•  loss  of  control  power 

•  low  hydraulic  pressure 

•  high  oil  temperature 

•  excessive  system  error 

•  operator's  command 

•  pilot's  command 

Shutdown  occurs  in  a  controlled  sequence. 
Solenoid  valves  close  to  trap  the  fluid  in  the 
cylinder.  Excess  pressure  is  bypassed  with 
relief  valves  across  the  cylinder  ports.  When 
the  system  stabilizes  the  condition  is  analyzed 
and  the  system  returned  to  a  safe  position  under 
manual  control . 

Level  3  emergency  systems  are  designed  to 
accommodate  the  most  severe  case  of  failure; 
the  complete  loss  of  electrical  and  hydraulic 
power.  For  this  case  a  set  of  accumulators  are 
provided  to  store  enough  energy  to  return  the 
system  to  a  safe  position.  The  initial  action 
is  to  trap  the  hydraulic  fluid  in  the  cylinders 
by  closing  fail-safe  solenoid  valves,  relieving 
excess  pressure  through  the  cross-connected 
relief  valves.  Also  included  are  automatic 
mechanical  devices  on  the  servovalve  spools  that 
control  the  pressure  in  the  accumulators  to 
"park"  the  system  with  all  cylinders  retracted. 
These  devices  are  programmed  to  move  the  most 
extended  cylinders  at  twice  the  velocity  of  the 
shorter  ones  to  avoid  any  hazardous  attitudes 
on  the  way  to  the  totally-retracted  position. 

SUMMARY 

In  summary,  we  have  described  the  specifica¬ 
tion  and  design  of  the  Rotorcraft  Simulator 
Motion  Generator  (RSMG).  The  system  is  intended 
to  replace  the  existing  6D0F  motion  generator  on 
the  NASA  Vertical  Motion  System  at  Ames  Research 
Center.  The  extended  capabilities  of  the  RSMG 
will  make  the  VMS  suitable  for  simulating  rotor- 
craft  as  well  as  fixed-wing  aircraft.  Its  per¬ 
formance  will  then  satisfy  the  requirements  of 
the  U  S.  Army's  long  range  rotorcraft  R&D  pro¬ 
grams. 

The  RSMG  is  a  4  degrees -of- freedom  (4D0F) 
motion  generator  to  be  mounted  on  the  200F 
Vertical  Motion  System.  To  fit  within  the  ex¬ 
isting  building,  the  RSMG  was  designed  as  a 
relatively  uncoupled  mechanical  system,  with 
independent  electronydraulic  actuators  for  each 
axis  of  motion.  The  control  systems  have  been 


designed  especially  to  maintain  stability  and 
performance  with  a  wide  range  of  payloads.  A 
computing  system  is  dedicated  to  the  RSMG  to 
aid  in  signal  handling,  subsystem  monitoring  and 
maintenance.  The  entire  RSMG  system  and  its 
test  subjects  are  protected  by  a  sophisticated 
3-level  safety  system  that  returns  it  to  a  safe 
attitude  In  the  event  of  any  foreseeable  mal¬ 
function. 

The  RSMG  promises  to  be  a  key  element  in 
the  utilization  of  the  VMS  to  Implement  the 
U.S.  Army's  Rotorcraft  Systems  Integration  Simu¬ 
lator  (RSIS). 
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ABSTRACT 

This  paper  discusses  fully  operational,  user-interactive  automated  training  equipment 
with  sophisticated  computer  graphics  and  video  that  can  be  packaged  into  hardware  no 
larger  than  a  suitcase.  Such  systems  have  been  assembled  using  thin-film  electro¬ 
luminescent  display  panels  and  transparent  touch-panel  overlays  coupled  with  interactive 
software  and  microcomputer  and  memory  technology.  The  display  quality  is  comparable  to 
or  better  than  that  of  most  CRT  installations  and  the  system  intelligence  can  exceed 
that  of  many  minicomputers.  Incorporated  touch  interaction  enables  users  to  be  facile 
with  its  operation  without  undergoing  undo  specialized  training.  Systems  of  this  type 
can  be  used  to  satisfy  a  great  majority  of  the  Services'  needs  for  sophisticated  training 
and  test  equipment,  while  being  completely  portable  and  usable  in  a  tactical  environment. 


INTRODUCTION 

The  continued  growth  of  battlefield  automated 
systems  and  generation  of  large  quantities  of  data, 
makes  it  essential  for  users  Vo  be  extremely 
facile  with  the  operation  of  such  systems  without 
undergoing  undo  specialized  training.  To  satisfy 
this  requirement,  while  maintaining  operator  con¬ 
trol,  such  systems  must  have  a  convenient  means 
for  the  user  to  communicate  with  the  data  base 
and  central  processor.  Also,  for  such  equipment 
to  be  useful  for  tactical  applications,  it  should 
be  highly  portable  and  rugged.  There  are  many 
definitions  of  portable,  but  in  this  context,  we 
refer  to  hardware  that  ia  easily  carried  by  one 
man.  There  have  been  a  nurber  or  demonstrations 
of  effective  computer  based  systems  which  meet 
good  user-interactive  standards  for  training 
purposes,  configured  in  relatively  large  installa¬ 
tions.  However,  the  type  of  system  described  here, 
has  comparable  capability,  while  entirely  packaged 
in  a  small  suitcase.  The  following  sections  will 
discuss  hardware  and  software  aspects  of  such  a 
system,  describe  a  sample  scenario  for  automated 
test  equipment  use  and  discuss  expected  further 

developments . 

SYSTEM  HARDWARE 

The  system  hardware  can  be  subdivided  into 
five  categories: 

(1)  Flat  panel  display 

(2)  Touch  interactive  mechanism 

(3)  Microcomputer  to  process  and  retrieve  infonna- 
tion,  control  the  display  and  respond  to  user 
requests. 

(4)  Easily  replaceable,  non-volatile  memory  to 
hold  specific  software. 

(5)  Miscellaneous  components  including  case, 
power,  supplies  and  the  like. 

The  following  subsections  will  discuss  the 
first  four  of  these  categories. 

Flat  Panel  Display 

High  quality  graphics  and  video  imagery  are 


achievable  using  thin-film  electroluminescent 
(TFEL)  display  technology.  Figure  1  depicts  a 
cross-sectional  view  of  a  TFEL  display.  The  dis¬ 
play  is  fabricated  on  a  single  sheet  of  glass 
using  inexpensive  vacuum  deposition  techniques. 
When  a  potential  of  approximately  200  volts  is 
placed  across  selected  row  and  column  electrodes, 
a  yellow-orange  emission  as  defined  by  the  elec¬ 
trode  intersection  is  seen  through  the  trans¬ 
parent  column  electrode.  By  utilizing  a  multi¬ 
plexing  (x-y  address)  partial  selection  scheme, 
one  can  address  the  entire  display,  row-at-a-time 
in  frame  times  comparable  to  standard  TV  rates 
(60  hertz  (Hz))  or  fastcr,  if  desired.  MOSFET 
technology  can  handle  the  required  voltage  levels 
and  integrated  versions  of  such  row  and  column 
drivers  are  available.  Live  television  as  well 
as  graphics  have  been  demonstrated  on  a  240  by 
320  pixel  resolution  display  panel  at  a  pixel 
spacing  of  68  lines-per-inch  (lpi) .  Various 
other  display  panels,  ranging  in  size  from 
inches  by  4  inches  at  80  lpi  to  10  inches  by 
12  inches  at  50  lpi  are  in  different  stages  of 
development.  A  100  lpi  panel  with  a  total  of 
512  by  640  pixels  is  near  completion. 

A  display  monitor  incorporating  the  240  by 
320  pixel  panel  dissipates  a  total  of  approxi¬ 
mately  10  watts  and  weighs  only  3  pounds.  Recent 
work  in  developing  new  drive  schemes  is  expected 
to  reduce  that  power  substantially.  Some  thermal 
tests  have  been  performed  and  indications  are 
that  complete  compliance  with  military  temperature 
specifications  is  expected.  Currently  used  epoxy 
seals  between  the  display  substrate  and  a  protec¬ 
tive  back  cover  glass  indicate  satisfactory  pro¬ 
tection  from  moisture,  however  hermetic  sealing 
techniques  are  presently  being  developed.  Life 
tests  performed  on  some  sample  displays  indicate 
operating  lives  in  excess  of  10,000  hours.  More 
complete  life  testing  is  being  conducted. 

Another  important  feature  of  these  displays  is 
contrast.  Referring  back  to  Figure  1,  it  will  be 
noted  that  there  is  a  black,  light  absorbing  layer 
behind  the  light  emitting  surface  and  in  front  of 
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the  rear  metallic  electrodes.  This  layer  absorbs 
in  excess  of  99.9  percent  of  the  light  lmplngent 
on  the  display,  resulting  In  a  high  degree  of 
legibility  at  relatively  low  luminance  levels. 
Legibility  in  a  direct  sunlight  environment 
(10,000  footcandles  (fc))  has  been  demonstrated 
with  only  IS  footlamberts  (fL)  luminance. 

Touch  Interactive  Mechanism 

The  incorporation  of  a  touch-interactive 
mechanism  over  the  display,  converts  it  from  a 
one-way  to  a  two-way  communicator.  The  effect  of 
the  touch  interactive  mechanism  is  to  allow  the 
operator  to  touch  the  surface  of  the  display 
screen  with  his  finger  or  other  passive  device, 
and  have  the  system  recognize  where  it  has  been 
touched.  By  correlating  this  touch  with  the 
displayed  information  at  that  point  and  with 
corresponding  data  or  processes  that  the  micro¬ 
computer  can  perform,  the  operator  can  query  or 
instruct  the  computer  via  the  touch.  It  is  not 
necessary  for  him  to  use  a  keyboard  or  any  other 
interactive  devices  such  as  buttons,  switches, 
light  pens,  or  the  like.  With  properly  written 
software,  the  touch  panel  can  make  the  system 
completely  self-instructing  since  the  display  can 
communicate  with  the  user  in  his  language  and 
remove  all  ambiguity  in  the  particular  operations 
that  the  user  has  to  take  through  the  provision 
of  various  types  of  menu  selection. 

A  number  of  touch  panel  mechanisms  are  possi¬ 
ble.  A  common  one  involves  the  incorporation  of 
two  transparent  layers  (glass  or  plastic)  with 
transparent  conductive  coatings  on  their  insides. 
The  pressure  of  a  touch  completes  electrical  con¬ 
tact  between  the  layers.  The  Central  Processing 
Unit  (CPU)  can  then  determine  where  the  touch 
took  place  based  either  on  the  transparent  con¬ 
ductor  pattern  or  on  a  resistance  calculation. 
Another  mechanism  involves  an  arrey  of  infrared 
(IR)  light  emitting  diodes  (LEDs)  and  correspond¬ 
ing  arrays  of  IR  detectors  around  the  periphery 
of  the  display.  The  finger  touch  then  breaks  the 
invisible  IR  array,  identifying  where  the  touch 
takes  place. 

In  a  later  section  we  describe  a  sample 
scenario  utilizing  touch  panel  operation. 

Microcomputer 

Microcomputers  have  sufficient  computing  power 
to  perform  training  and  test  functions  and  drive 
the  display  while  keeping  cost  and  power  consump¬ 
tion  low.  The  one  chosen  should  have  at  least  a 
16  bit  word  length  to  enable  it  to  perform  fast 
graphics  on  the  display  and  for  rapid  access  to 
the  mass  storage.  On-board  Random  Access  Memory 
(RAM)  should  be  capable  of  storing  the  operating 
system  software  and  programs  necessary  to  inter¬ 
pret  the  operator's  commands  to  access  more 
information  from  the  mass  storage  device.  There 
should  be  sufficient  input  and  output  to  inter¬ 
face  to  the  display  and  the  mass  storage  device. 

There  are  a  number  of  commercially  available 
microcomputer  boards  available  that  adequately  meet 
the  above  criteria. 

The  boards  with  16  bit  CPUs  typically  contain 


on  the  order  of  32  kilobytes  (Kbytes)  of  read  only 
memory  (ROM)  ,  64  Kbytes  RAM  >  and  serial  and 
parallel  Input/Output  (I/O)  ports  on  a  single 
moderately  sized  circuit  beard.  This  is  suffi¬ 
cient  to  meet  expected  needs  for  the  type  of 
equipment  being  discussed,  other  than  the  mass 
storage  which  will  be  discussed  in  the  next 
section.  It  is  not  the  intent  of  this  paper  to 
suggest  a  particular  CPU,  memory  or  software 
architecture,  but  simply  to  point  out  that  suffi¬ 
cient  processing  and  memory  capability  is  present¬ 
ly  available  in  components  sufficiently  small  to 
be  completely  packaged  in  a  suitcaae-sized  con¬ 
tainer,  including  the  display.  As  the  micro¬ 
computer  technology  evolves  over  the  next  few 
years,  the  capabilities  will  grow  further,  while 
power  dissipation  and  size  shrink.  Exemplary  of 
this  expectation  is  the  Military  Computer  Family 
single-board-computer,  which  will  execute  the 
highly  structured  language  Ada,  have  a  32-bit  CPU 
and  dissipate  only  5  watts,  all  on  a  single  board. 

Memory 

The  mass  storage  device  must  be  capable  of 
storing  all  of  the  information  needed  for  the 
particular  task  being  performed,  such  as  field 
testing  of  a  piece  of  equipment. 

In  the  case  of  automated  testing  of  battle¬ 
field  equipment,  it  is  necessary  that  we  have 
stored  all  of  the  pertinent  test  and  diagnostic 
information  about  that  oystem.  Depending  upon  the 
complexity  of  the  equipment  to  be  tested  and  the 
level  of  testing  to  be  performed,  this  informa¬ 
tion  would  typically  occupy  the  equivalent  of  a 
number  of  field  manuals,  although  in  certain 
cases  much  less  information  than  that  would  be 
required. 

For  less  complex  tasks,  storage  of  the  order 
of  Ij— 1  megabytes  (Mbytes)  of  information  could  be 
satisfactory.  This  is  readily  accommodated  with 
semiconductor  or  magnetic  bubble  memory  occupying 
the  equivalent  of  one  or  two  circuit  boards. 
Because  of  the  required  versatility  of  the  pro¬ 
posed  equipment,  it  would  be  packaged  as  a  plug¬ 
in  module.  For  the  more  complex  tasks,  such  as  in 
the  example  given  in  the  next  section,  memories  in 
excess  of  10  Mbytes  could  be  required.  This  may 
be  provided  by  magnetic  disc  technology  which  can 
now  store  50  Mbytes  in  ?  5  inch  by  9  inch  by  14 
inch  package  weighing  20  pounds.  Optical  disc 
technology  can  store  up  to  500  Mbytes  in  a 
slightly  larger  package.  Both  of  these  tech¬ 
nologies  are  evolving  to  smaller  physical  sizes 
In  more  rugged  packages. 

SYSTEM  SOFTWARE 

The  software  required  to  support  this  equip¬ 
ment  falls  into  two  ectegories.  The  first  is 
that  coftware  which  is  the  same,  regardless  of 
the  particular  applications  beir.g  performed  (a.g., 
independent  of  equipment  being  tested)  .  When 
contact  Is  made  to  the  touch  panel,  the  transla¬ 
tion  of  that  touch  to  an  x-y  coordinate  is  done 
with  this  type  of  software.  Once  an  x-y  pair  is 
calculated,  the  computer  can  make  a  (control) 
decision  based  on  it.  The  choices  that  the 
computer  has  to  pick  from  may  be  in  the  form  of 
a  menu.  The  selection  of  one  choice  from  the  menu 
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is  zoned  to  a  specific  window  of  x-y  coordinates 
(e.g.,  50  <  x  <  60,  110  <  y  <  120  might 
define  the  PROCEED  function).  The  definitions 
of  these  choices  and  their  windows  fall  into 
the  second  category  (equipment  dependent) . 

Also  in  the  first  category  (equipment 
independent)  is  the  software  that  controls  the 
display.  As  described  in  the  hardware  section 
above,  the  electroluminescent  (EL)  display  is 
matrix  addressed  (point-by-point).  In  order 
to  use  the  display,  a  means  of  accessing  one 
pixel  element  and  turning  it  or  or  off  is 
needed.  Also,  a  means  of  providing  end  points 
for  line  segments,  and  having  the  line  appear 
on  the  display  is  necessary.  In  addition,  a 
character  set  for  textual  messages  must  be  pro¬ 
vided.  Because  these  features  are  required  in 
all  equipment  tests,  they  also  fall  into  the 
first  category.  All  of  the  software  in  this 
category  would  reside  in  a  small  (approximately 
8  Kbytes)  amount  of  ROM  that  would  be  permanently 
placed  in  the  unit. 

The  second  category  software  depends  on  the 
particular  equipment  being  tested,  and  therefore 
is  resident  in  the  removable  storage  referred  to 
in  the  previous  section.  It  could  require 
millions  of  bytes.  This  removable  software 
package  (or  personality  module)  would  hold  the 
data  on  measurements  to  be  made,  their  nominal 
values,  and  the  graphics  of  the  various  stages 
to  be  shown  on  the  display,  as  well  as  the 
instructions  indigenous  to  that  particular 
piece  of  equipment. 

SAMPLE  SCENARIO 

As  an  example  of  how  such  a  system  might 
function,  we  include  a  simulation  of  a  possible 
test  sequence  for  failure  isolation  in  an 
AN/PRC-77  FM  transceiver.  We  are  assuming  a 
airect  support  activity  where  an  operator  is 
using  his  suitcase-sized  system  to  analyze  the 
equipment  being  tested  in  a  relatively  hostile 
environment.  The  figures  are  actual  photographs 
of  the  240  by  320  flat  panel  display  mentioned 
earlier.  The  operator  turns  his  aystem  on  and 
is  presented  a  menu  selection  asking  him  to 
identify  (by  touching  the  proper  word)  the 
particular  unit  under  test  (UUT) .  The  menu 
selection  presented  to  him  would  include  what¬ 
ever  system  software  is  presently  in  his  auto¬ 
matic  tester,  or  might  ask  Mm  to  Insert  a 
memory  element  corresponding  to  that  unit. 

Figure  2  is  a  photograph  of  the  next  display  to 
appear,  which  Instructs  the  operator  by  words 
and  graphically,  how  to  connect  his  automatic  test 
equipment  to  the  UUT.  rfhen  he  has  completed 
initial  hookup,  he  is  instructed  to  press 
PROCEED.  In  developing  such  a  scenario,  it  is 
Important  to  realize  that  there  is  complete 
flexibility  as  to  the  location  and  size  of  the 
"touch"  boxes  as  well  as  the  nature  of  the  alpha- 
numerics  and  graphics.  This  means  that  while  the 
particular  test  equipment  is  being  developed, 
user  feedback  and  human  factors  considerations 
can  strongly  influence  the  final  configurations 
without  undo  cost.  Continuing  with  the  scenario. 
Figures  3  and  4  step  the  operator  through  visual 
examination  of  filter  FL3  and  Terminal  Block  A57, 
as  indicated  by  Instructions  and  cursor. 

Figure  5  begins  instruction  on  probe  testing  and 
at  Figure  6,  the  test  equipment  notifies  the 


operator  that  a  failure  has  been  detected  and 
instructs  him  to  proceed  to  the  module  level 
test.  Not  taking  anything  for  granted,  the  oper¬ 
ator  is  then  Instructed,  as  shown  in  Figure  7, 
how  to  open  the  defective  module,  and  to  tell  the 
computer  when  he  has  finished  this  task.  Figure 
8  then  presents  to  the  operator,  a  drawing  of  the 
circuit  board  that  he  is  confronted  with.  Sub¬ 
sequent  frames  will  step  him  through  testing  the 
various  components  until  the  failure  is  localized. 
When  it  is  necessary  for  the  operator  to  commu¬ 
nicate  alpha  symbols  or  numeric  readings  to  the 
computer,  his  choice  of  symbols  or  a  number  pad 
will  appear  on  the  display,  for  touch  interaction. 
Obviously,  with  a  scheme  like  this,  it  iBn't 
necessary  for  the  operator  to  make  any  signifi¬ 
cant  decisions,  nor  understand  anything  about 
ADP  equipment  other  than  the  location  of  the 
on-off  switch. 

CONCLUSIONS 

We  have  described  the  critical  elements  of  a 
suitcase-sized  system  that  could  be  a  vital 
element  for  training  and  testing  on  battlefield 
automated  systems.  There  is  nothing  original 
about  the  mode  of  operation  of  such  a  system 
other  than  the  fact  that  it  can  be  performed, 
with  presently  available  hardware,  in  a  very 
small  package.  With  conventional  technology,  the 
display  alone  would  occupy  a  larger  package  size. 
We  are  convinced  that  the  hardware  and  operations 
described  here  can  be  revised  many  times  over  by 
Interested  workers,  offering  substantial  improve¬ 
ments,  and  hope  that  this  paper  might  act  as  a 
stimulus  in  that  regard. 
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ELECTROLUMINESCENT  LAYER 


Figure  3.  Sample  Scenario  Frame  No.  2  Filter  Examination 


Terminal  Block  Examination 


Frame  No.  7  PC  Board 
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ABSTRACT 

The  Department  of  Defense  (DOD)  is  speeding  an  estimated  $6  billion  on 
software  each  year,  and  this  budget  is  forecasted  to  increase  by  15  to  20 
percent  annually.  In  recognition  of  this  software  cost  spiral  and  to  halt 
the  proliferation  of  computer  programming  languages,  DOD  has  sponsored  the 
development  of  a  new  language--called  Ada--to  provide  a  standard, 
computer-independent  high  order  language  for  major  defense  systems  software. 
Ada  is  expected  to  have  widespread  application  throughout  DOD  and  will 
require  training  for  thousands  of  individuals.  Control  Data  is  developing  a 
solution  to  the  Ada  training  problem  using  the  Control  Data  PLATO® 
computer-based  education  system. 


THE  PROBLEM 

The  U.S.  Department  of  Defense 
initiated  Ada,  the  new  high  order 
language  (HOL),  in  1976  with  the 
objective  of  eventually  standardizing 
programming  languages  into  one  HOL. 

The  five-year  design  effort  involved 
defense  departments,  academia  arid 
industry  from  fifteen  countries.  Ada 
is  currently  going  through  American 
National  Standards  Institute  procedures 
to  become  a  U.S.  standard  and  is 
expected  to  become  an  International 
standard  through  the  International 
Organization  for  Standardization. 

There  is  widespread  interest  In  Ada, 
and  it  is  believed  that  a  requirement 
for  it  will  evolve  in  commercial  as 
well  as  defense  applications. 

DOP’s  Software  Problem 

DOD  is  spending  an  estimated  $6  billion 
on  software  each  year  and  as  indicated 
in  figure  1,  the  software  budget  is 
forecasted  to  increase  rapidly.  (1) 

In  recognition  of  this  accelerating 
software  cost  spiral,  DOD  established 
policy  framework  (DOD  Directive 
5000.29)  in  1976  for  advanced  computer 
technology,  including  the  requirement 
for  a  DOD  approved  HOL.  The  resulting 
development  of  Ada  is  now  history,  and 
the  design  of  Ada  culminated  in  the  DOD 
"Ada  Debut"  on  September  4,  I960. 

Ada  began  in  000  as  one  of  the 
initiatives  aimed  at  controlling  the 
"embedded  computer"  software  costs. 
Embedded  computers  are  those  equipments 


Figure  1.  Software  budget  forecast 


procured  as  part  of  weapons  systems  and 
controlled  by  the  DOD  5000. XX  series  of 
directives,  Ada  is  initially  being 
hosted  on  automatic  data  processing 
computers  used  in  support  of  weapons 
systems.  Furthermore,  Ada  is  perceived 
as  being  a  system  design  language  which 
opens  the  door  for  training  of  hardware 
and  system  designers,  program  managers, 
as  well  as  software  designers. 

As  indicated  in  figure  2,  the  costs  of 
computer  hardware  are  continuing  to 
decrease,  but  software  costs,  being 
labor-intensive,  are  rapidly 
increasing.  A  sizable  portion  of 
software  life-cycle  costs  can  be 
attributed  to  maintenance  snd  training. 
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Figure  2.  Hardware  vs.  software 
cost  forecast 


A  fundamental  problem  confronting 
industry  and  DOD  is  the  shortage  of 
computer  programmers.  Figure  3  shows 
that  the  number  of  computers  in  the 
U.S.  is  growing  much  faster  than  the 
number  of  programmers  and, 
unfortunately,  programmer  produc¬ 
tivity  is  not  significantly  improving. 


Figure  3.  Growth  rate  of  programmers 
vs.  computers 


The  software  labor  shortfall  is  likely 
to  get  worse.  There  is  an 
industry-wide  need  to  recruit  and  train 
more  computer  people. 

The  Ada  Training  Problem 

More  and  more  computers  will  be  used  by 
000,  requiring  more  application 
software.  Adding  this  to  the  labor 
shortfall  problem,  and  the  rapidly 
escalating  cost  of  software  resources, 
makes  the  need  for  training  apparent. 


At  least  four  classes  of  education  and 
training  are  needed  to  successfully 
implement  Ada  throughout  DOD: 

1.  An  overview  of  Ada  must  be 
provided  for  managers  and  executives 
and  as  a  first  course  for  software  and 
systems  personnel. 

2.  A  retraining  course  for 
programmers  who  are  already 
knowledgeable  on  another  HOL,  such  as 
FORTRAN,  is  required. 

3.  !\  complete  training  pregram 
from  computer  fundamentals  through  the 
Ada  language  is  needed  for  the 
thousands  of  computer  neophytes  who 
must  be  trained  to  relieve  the  labor 
shortfall  problem  facing  industry  and 
00D. 


A.  Unique  application  training  on 
how  to  use  the  Ada  HOL  effectively  In 
Solving  specific  computer  system 
requirements  must  be  provided. 


THE  SOLUTION 

Control  Data  is  a  leader  in 
computer-baseo  education  ana  is 
currently  providing  hundreds  of  courses 
via  the  Control  Data  PLATO  system. 
Control  Data  is  planning  PLATO-hased 
courseware  for  Aaa  in  three  types  of 
courses.  The  first  will  address  the 
"whats  and  whys"  of  Ada  ana  will 
culminate  in  an  Overview  of  Ada 
course.  The  second  course  wiTT  address 
the  "how  to’s"  of  Aaa  and  will  be  a. 

Ada  Programming  Fundamentals  course 
basec  on  the  PLATO  system.  'The  third 
area  of  training  will  actually  be  a 
series  of  courses  designed  to  provide 
000  application  unique  training,  such 
as  "How  to  use  Ada  effectively  in 
solving  a  digital  signal  processing 
problem. " 

These  courses  will  satisfy  the  four 
classes  of  Ada  training  previously 
loentified. 

Individualized  Instruction 


Unlike  traditional  lecture-based 
instruction,  where  a  single  instructor 
speaks  to  a  captive  group  of  students, 
individualized  instruction  allows 
students  to  learn  independently  from 
sets  of  structured  learning  resources. 
Each  learning  resource  is  a  carefully 
designed  package  of  instructional 
materials  presented  by  one  or  more 
types  of  media  such  as 
computer-assisted  Instruction,  text, 
and  audiovisual  products  (audiotapes, 
videotapes,  filmstrips,  or  slide 
shows).  These  materials  communicate 
the  same  knowledge  presented  by 
instructors  through  lectures. 
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Extensive  research  by  educators  and 
psychologists  proves  the  importance  of 
interaction  to  improve  learning  and 
strengthen  retention.  Multimedia 
individualized  instruction  checks 
students'  understanding  just  as 
instructors  do--by  asking  questions. 
Strategically  placed  questions  or  other 
response  requirements  elicit 
interactions  from  each  student.  The 
purpose  of  the  interactions  may  be  to 
check  for  understanding  of  concepts, 
require  relationships  to  be  drawn 
between  concepts,  emphasize  important 
points,  illustrate  rules,  or  simulate 
procedures . 

Students  progress  at  their  own  rate  as 
they  work  through  each  learning 
activity.  Because  the  burden  of 
delivering  information  is  shifted  from 
the  instructor  to  the  learning 
activities,  instructors  can  give 
individual  attention  to  each  student. 
Instructors  are  freed  to  answer 
specific  questions,  provide 
encouragement,  provide  additional 
explanations  to  meet  unique  needs  of 
individual  students,  and  get  to  know 
each  student  on  a  personal  basis. 
Students  also  receive  immediate 
feedback  on  how  well  they  are 
learnina--they  are  not  made  to  wait  for 
quizzes  given  at  arbitrary  times 
whether  or  not  they  are  ready. 

Numerous  benefits  are  gained  from 
individualized  instruction.  Folloving 
are  some  of  the  more  important  benefits: 

•  Because  students  progress  at 
their  own  rate,  fast  students 
are  not  held  back  and  slow 
students  do  not  fall  behind. 

•  Training  to  the  desired  skill 
levels  is  more  precise.  Better 
students  are  not  overtrained  and 
slow  students  are  nut 
undertrained . 

•  Instructors  can  spend  more  time 
with  students  who  need  special 
assistance  rather  than 
attempting  to  teach  to  an 
idealized  average,  or  slightly 
below  average,  student. 

•  Students  are  more  active.  They 
cannot  sit  passively  while 
instruction  goes  on  around 
them.  This  interaction  improves 
learning  and  increases  retention 
for  each  student. 


•  Attrition  can  be  reduced  because 
students  are  not  forced  to  learn 
at  an  externally  imposed  pace 
they  may  be  unable  to  meet. 

More  attention  can  be  given  to 
specific  learning  problems 
unique  to  individual  students; 
this  decreases  chances  of 
failure  resulting  from 
difficulty  in  only  one  area. 

•  Deficiencies  of  the  training 
program  itself  can  be 
identified,  diagnosed,  and 
corrected  quickly  as  a  result  of 
the  concrete  structure  of 
modularized  learning 
activities.  The  result  is  more 
cost-effective,  efficient 
training. 

•  More  precise  control  of  training 
is  possible  because 
instructional  materials 
contained  in  learning  activities 
are  not  subject  to  instruc¬ 
tors'  reassignment,  illness, 
lack  of  training,  or 
forgetfulness. 

In  computer-managed  instruction  (CMI), 
the  functions  of  testing,  study 
assignments  (prescriptions),  and  record 
keeping  are  assumed  by  the  computer. 

In  this  application,  the  computer 
becomes  a  powerful  tool  for  relieving 
instructors  of  administrative  tasks. 

In  CMI,  all  tests  are  administered 
on-line  (on  a  PLATO  terminal).  This 
on-line  testing  enables  the  computer  to 
immediately  score  tests,  analyze  item 
responses,  and  store  the  data  in  files 
containing  complete  histories  of 
students'  past  performances  and 
assignments.  Decision  logic  tables,  or 
algorithms,  programmed  into  thp 
computer  can  then  use  this  pool  of 
information  to  make  prescriptions 
tailored  to  students'  needs.  These 
prescriptions  can  assign  remedial  work, 
more  advanced  lessons,  additional 
tests,  outside  references  or  resources, 
or  any  one  of  a  number  of  instructional 
alternatives  desired  by  an  instructor. 

Records  of  learning  performance  stored 
in  student  data  files  can  be  examined, 
sorted,  and  arranged  in  any  format 
desired  by  an  instructor  or  training 
manager.  This  data  may  ii, elude 
individual  scores  for  a  given  student 
per  assignment,  summary  scores  for  all 
students  on  a  given  test  or  learning 
activity,  number  of  times  a  particular 
activity  is  assigned,  time  elapsed 
between  tests,  and  total  hours  in 
training. 
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The  benefits  of  CMI  thus  lie  in  the 
precision,  control,  and  reduced 
instructor  burden  it  provides.  Each 
assignment  is  constructed  from  a  known 
data  base.  Therefore,  if  something  is 
wrong  with  a  particular  assignment  or 
test,  it  can  be  traced  to  the  logic 
behind  it  and  corrected.  In 
traditional  management  modes,  decisions 
are  often  based  upon  incomplete 
information  or  best  guesses.  These 
subjective  decisions  make  it  difficult 
to  systematically  improve  a  training 
program.  A  CMI  program,  however,  can 
be  steadily  improved  by  building  upon 
objective  data  describing  each 
student's  experience.  The  mass  of  data 
available  can  be  manipulated  on  the 
computer  in  a  matter  of  seconds,  while 
it  may  take  hours  or  days  to  do  so 
manually.  The  chances  of  errors  in 
transcription  are  also  greatly 
reduced.  This  is  very  important  when 
dealing  with  test  results,  where  an 
error  could  mean  the  difference  between 
success  or  failure. 

In  short,  CMI  automates  most  of  the 
mechanical  tasks  of  managing  a  training 
program.  This  automation  not  only 
enables  instructors  to  spend  more  time 
with  their  students,  it  also  increases 
control  over  the  training  environment 
and  improves  accuracy  and  usefulness  of 
essential  training  information. 

The  Control  Data  PLATO  System 

The  Control  Data  PLATC  system  is  an 
interactive  computer-based  education 
system,  which  consists  of  hardware, 
software,  a  computer  system,  and 
courseware.  The  hardware  portion 
consists  of  graphic  display  terminals 
and  a  communications  network.  The 
software  portion  runs  as  an  application 
package  under  the  control  of  the 
Network  Operating  System  .  This 
software  supports  the  PLATO 
communications  equipment,  manages 
computer  resources  available  at  the 
central  site,  and  processes  all 
interaction  between  the  central  system 
and  the  PLATO  terminal.  The  PLATO 
system  runs  on  the  CDC®  CYBER  170  or 
COC®  CYBER  70  computer  systems.  Other 
PLATO  systems,  such  as  Micro  PLATO,  can 
be  used  for  courseware  delivery  in  a 
stand-ralone  mode  and  connected  to  the 
central  PLATO  system  for  authoring 
capability . 

Overview  of  Ada 

The  first  course,  Overview  of  Ada,  is 
intended  to  provide  programmers  and 
their  managers  with  an  understanding  of 
the  development  and  benefits  of  this 
new  programming  language.  The  course 
presents  the  rationale  behind  the 
design  of  the  Ada  language  and  ?  brief 
history  of  its  development,  as  well  as 
the  economic,  productivity,  anc 
reliability  problems  in  software 


development.  Each  of  the  principles 
that  were  applied  in  an  attempt  to 
overcome  the  problems  and  produce  more 
efficient  and  reliable  software 
development  is  explained,  along  with 
some  of  the  language  features  that  were 
developed  to  achieve  these  principles. 
The  purpose  of  this  course  is  to 
introduce  the  Ada  language  so  that  It 
is  viewed  as  a  viable  alternative  to 
other  programming  languages. 

The  introduction  illustrates  the 
current  economic  problems  faced  in 
software  development  and  shows  why  the 
design  and  development  of  a  new 
language  were  undertaken.  It 
highlights  the  history  of  the  language, 
touching  on  key  events  and  emphasizing 
the  computing  principles  that  were 
applied  In  this  effort  to  provide 
greater  productivity  and  reliability 
through  the  use  of  a  new  language. 

The  course  explains  each  of  these 
computing  orinciples  intuitively, 
illustrating  how  its  application  can 
facilitate  software  development.  The 
features  designed  to  support  the 
principle  are  also  presented.  For 
example,  modularity  Is  one  of  the 
principles  that  was  applied  in  the 
design  of  the  Ada  language.  The 
concept  of  modularity  is  discussed,  and 
the  language  feature — packages--that 
supports  this  principle  is  presented, 
along  with  the  benefits  that  packages 
provide,  such  as  permitting  the 
division  of  labor  and  providing  more 
readable  code. 

The  course  discusses  ether  principles 
incluoing  reusable  software,  separate 
compilation,  tasking,  and  strong 
typing.  The  features  that  were 
incorporated  in  the  language  to  achieve 
these  principles  are  also  presented. 

Finally,  there  is  a  discussion  of  the 
rationale  for  the  design  and 
development  of  an  Ada  environment.  The 
functions  that  the  Ada  Programming 
Support  Environment  will  provide  and 
the  different  levels  of  the  environment 
are  explained. 

Ada  Programming  Fundamentals 

The  second  course,  Ada  Programming 
Fundamentals,  is  Intended  to  provide 
the  experienced  programmer  with  an 
Introduction  to  the  Ada  language. 
Overview  of  Ada  is  a  prerequisite  to 
this  course,  providing  a  background  in 
the  basic  concepts  underlying  the 
language.  The  purpose  of  Ada 
Programming  Fundamentals  Is  to  teach 
the  Ada  language  to  experienced 
programmers  in  a  manner  that  achieves 
the  full  benefits  of  the  new  language. 
Teaching  the  syntax  of  the  Ada  language 
is  not  adequate  preparation  for 
programming  with  the  language. 

Instead,  many  programmers  need  to  learn 
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a  new  discipline  of  designing  and 
developing  programs  to  achieve  the 
desired  benefits  from  the  language. 

The  use  of  a  top-down  modular  approach 
in  designing  program?  enhances  the 
effective  use  of  the  Ada  programming 
language.  A  section  of  this  course 
discusses  top-down  modular  design  of 
Ada  programs  and  the  use  of  the  limited 
set  of  structured  control  structures  in 
designing  algorithms.  The  remainder  of 
the  course  emphasizes  these  approaches. 

The  intention  of  this  course  is  to 
provide  the  experienced  programmer  with 
a  working  knowledge  of  the  Ada 
language,  so  that  it  is  possible  to  use 
it  to  write  programs.  The  course  is 
not  intended  to  cover  all  variations  of 
every  language  feature,  but  instead 
prepares  the  programmer  for  getting 
started  using  the  language. 

An  Exampto  of  a  PLATO  Court* 

The  addendum  to  this  paper  contains 
screen  prints  made  directly  from  the 
PLATO  terminal  showing  a  lesson 
sequence  from  the  Ada  Programming 
Fundamentals  course"!  The  lesson  is  on 
the  Ada  "IF"  statement  and  the  screen 
prints  show  the  following: 

•  Lesson  title 

•  Objectives  for  the  lesson 

•  Flow  charts  of  control  selection 
a  Explanation  of  "IF”  statement 

syntax 

•  Example  of  "IF"  statement 

•  Exercise  using  "IF"  statement 

The  use  of  PLATO  graphics  greatly 
enhances  the  learning  potential  of  the 
CRT,  or  display,  terminal  by  providing 
a  “picture  drawing"  tool  for  the 
author.  Students  assimilate  ner 
concepts  more  effectively  through 
graphics  combined  with  text. 

Reference* 

(1)  Figures  1,  2,  and  3  are  from 
the  DOD  Digital  Data 
Processing  Study  performed  by 
an  industry  team  chaired  by 
Mr.  David  G.  Stephan,  Control 
Data  Corporation, 

Minneapolis,  Minnesota.  The 
study  was  performed  under  the 
auspices  of  the  Electronic 
Industries  Association. 


<  iU  Win:  i 
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Screen  Print  1:  Lesson  title 
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INTRODUCTION 


The  purpose  of  this  activity  is  to  familiarize 
you  with  the  if  statement,  its  use  and  its 
syntax. 


fit  the  end  of  this  activity  you  should  be  able 
to: 

o  Identify  when  an  if  statement  should  be  used 
as  a  control  structure. 

o  Code  the  if  statement  with  two  branches. 

o  Code  the  if  statement  with  multiple  branches. 


Press  NEXT  to  continue 


Screen  Print  3,  Display  1:  Plow  charts  of  control  selection 


The  order  of  execution  of  statements  in  a  program 
is  generally  sequential.  The  execution  proceeds 
statement  by  statement  through  the  program.  There 
are  circumstances  where  it  is  necessary  to  alter 
this  sequential  flow  of  the  program.  Conditional 
statements  permit  the  choice  between  several  courses 
of  action  depending  on  whether  a  specified  condition 
is  met.  If  the  condition  is  met,  one  sequence  of 
statements  is  executed. 


Screen  Print  4:  Explanation  of  "IF"  statement  syntax 


The  syntax  of  the  i f  statement  is: 

if  condition  thin 

sequenoe.o  f .statement  a_  i 

«lM 

sequence.o  f .statement s_2 

tnd  ifi 

When  the  condition  is  met,  the  first  sequence 
statements  is  executed,  and  when  it  is  not  the 
second  sequence  is  executed. 

The  condition  is  any  Boolean  expression,  such 

x  <■  a. 


Press  NEXT  to  continue 


Screen  Print  5,  Display  1:  Example  of  "IF"  statement 


AN  EXfflPLEi 

This  is  an  example  of  an  Rda  function  that  usee  the 
if  statement  to  return  the  values,  TRUE  or  FALSE, 
depending  on  whether  a  weight  is  within  a  given 
range. 


function  CHECKJNEXGHT  (Hi HEIGHT)  return  BOOLEAN  1» 

begin _ ' _ 

if  H  in  HEIGHT  rangeMB. ,  411 
return  TRUE. _ 

else 

return  FALSEi 
end  ifi 

end  CHECK.NEIGHTi 


In  this  function,  if  the  weight  H  is  within  the  acceptable 
range,  then  the  function  returns  the  value  TRUE. 


Press  NEXT  to  conti  me 


Screen  Print  5,  Display  2 


AN  EXfflPLEi 

This  is  an  example  of  an  Ada  function  that  uses  the 
if  statement  to  return  the  values,  TRUE  or  FALSE, 
depending  on  whether  a  weight  is  within  a  given 
range. 

function  CHECKJEIGHT  <H«  HEIGHT)  return  BOOLEAN  ia 
begin 

if  H  in  HEIGHT  range  390.. 41B 
return  TRUEi 

Hiss  ^ 


return  FALSEi 


In  this  function,  if  the  weight  H  is  within  the  acceptable 
range,  then  the  function  returns  the  value  TRUE. 

If  the  weight  is  not  within  the  acceptable  range,  then  the 
function  returns  the  value  FALSE. 


Press  NEXT  to  continue 
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fi,***^  . . .  *'  ' 


Display  2 
removes 
first  box 
•  and  then 
boxes  the 
lower 

information. 

j 


Display  2 
adds  this 
sentence. 


Screen  Piint  6,  Display  Is  Exercise  using  "IF"  statement 


You  will  assist  in  writing  a  function  that  computes 
daily  pay.  If  the  day  worked  is  either  Saturday  or 
Sunday,  then  the  pay  computed  should  be  1.5  times  the 
normal  wage. 

function  COMPUTE  PAY  (H1HOURS1  MiNAGEi  TODAY* DRY) 
return  DOLLAR?  is 
begin 

if  TODAY  in  SATURDAY. .SUNDAY 
return  1.3  *  H  *  Hi 

return  H  *  M 

end  CONPUTE.PAYi 


I - 

Enter  the  keyword  that  follows  the  condition  in  an  if 
statement . 
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Screen  Print  6,  Display  2 


AN  EXERCISEi 

You  will  assist  in  writing  a  function  that  cornputcs 
daily  pay.  If  the  day  worked  is  either  Saturday  or 
Sunday,  then  the  pay  computed  should  be  1.5  times  the 
normal  wage. 

function  CONPUTEPAV  (H1HOURS1  N1HAGE1  TOOAYiQAV) 
return  DOLLARS  is 
bog  in 

if  TODAY  in  SATURDAY.. SUNDAY  then 
return  1.5  *  H  *  Hi 

^  return  H  *  N 

end  COMPUTE JPAYi 


Enter  the  keyword  that  precedes  the  second  sequence  of 
statements  in  an  if  statement. 


Upon 

completion 

of  first 

direction, 

second 

direction 

appears. 
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ABSTRACT 

Integration  of  system  software  and  hardware  is  without  a  doubt  the  system  development 
activity  which  enjoys  the  greatest  management  visibility.  It  is  during  this  phase  of  system 
development  that  past  3lns  of  superficial  analysis  and  design  of  both  software  and  hardware 
surface  to  be  seen  by  all  and  where  system  completion  often  appears  to  follow  an  asymptotic  course 
to  infinity. 

This  paper  provides  a  brief  summary  of  the  software  development  practices  followed  in 
implementing  the  software  for  the  Multi-Environment  Trainer  (MET),  a  complex  multi-computer  naval 
ship  trainer,  and  an  in-depth  discussion  of  the  procedures  used  to  direct  the  day-to-day 
software/hardware  integration  activity.  Actual  experiences,  good  and  bad,  are  discussed  and 
findings  expressed  in  terms  of  problem  solutions  and  recommendations. 


INTRODUCTION 
System  Description 

The  MET  system  is  a  tactical  team/subteam 
trainer  designed  to  simulate  multiple-ship 
tactical  operations.  The  functions  being 
simulated  3pan  the  under  sea,  surface  and  air 
environments  and  include  operations  in  the 
Bridge,  Combat  Information  Center  and  Sonar 
areas.  The  MET  trainer  is  designed  about  the 
use  of  eight  computers  (SEL  32/7S)  configured  in 
two  groups  of  four  processors,  see  Figure  1. 
Each  group  of  four  computers  performs  a  set  of 
dedicated  functions  in  real-time  which  simulate 
the  operation  of  a  single  PCG  or  PGG  class  ship. 
Data  exchange  among  the  eight  computers  is 
accomplished  via  a  block  of  shared  memory,  while 
access  to  required  peripherals  is  controlled  by 
a  program  directed  peripheral  switch. 

Software  Description 

The  MET  software  architecture  is 
characterized  by  five  subsystem  areas:  Master 
Control,  Passive  Sonar,  Active  Sonar,  Radar  and 
Off-Line  Processes.  The  first,  four  subsystem 
areas  represent  the  real-cirae  portion  of  the  MET 
software  which  resides  in  each  of  the  two  sets 
of  four  MET  computers.  The  remaining  software, 
which  does  not  execute  in  real-time,  is  referred 
to  as  Off-  Line  Processes  and  includes  such 
programs  as  Diagnostics,  Daily  Readiness,  Data 
Base  Generators, etc.  Execution  of  the  MET 
software  resident  in  each  computer  is  directed 
by  a  simulation  executive  which  operates  under 
control  of  the  SEL  Real-Time  Monitor  (RIM) 
operating  system.  Computations  performed  by  all 
of  the  MET  computers  generate  va’ues  which 
contribute  to  a  large  and  resonably  complex 
system  data  base  which  is  made  available  to  each 
of  the  processors  in  real-time.  The  MET 
programs,  with  very  few  exceptions,  have  all 
been  written  using  FORTRAN-77. 

One  of  the  more  challenging  aspects  of  the 
MET  software  has  been  the  fact  that  it  interacts 
with  a  large  complement  of  operational  shipboard 
equipment,  e.g.  ship  control  consoles,  sonar 


control  consoles,  bathythermographic  and  depth 
recorders,  torpedo,  mk-75  gun  and  Phalanx  fire 
control  consoles,  SPS-40B  and  55  radar  consoles, 
MR-92  fire  control  consoles  and  various  and 
sundry  analog,  synchro  and  discrete  devices. 

Software  Status 

At  the  time  of  this  writing,  implementation 
of  the  MET  software  has  been  completed  and  all 
program  modules  have  been  unit-tested  and 
submitted  for  integration.  The  software/hard¬ 
ware  integration  activity  i3  in  its  final  stages 
and  the  system  is  rapidly  approaching  customer 
acceptance  testing. 

SOFTWARE  DEVELOPMENT 
Software  Concepts 

Before  getting  into  the  discussion  of 
software/hardware  integration,  which  is  the 
principal  topic  of  this  paper,  I  believe  it  will 
be  germane  to  the  discussion  to  briefly  review 
the  procedures  which  were  adopted  and  enforced 
during  the  MET  software  design  and 
implementation  phases.  Software  development 
was  accomplished  by  establishing  Chief 
Programmer  Teams  for  each  of  the  five  subsystem 
areas.  Each  of  the  teams  was  fully  responsible 
for  the  design,  implementation  and  test  of  all 
the  software  required  by  a  particular  subsystem 
and  each  contributed  one  or  two  of  its  members 
to  participate  on  the  software/hardware 
integration  team. 

Devc’opment  of  the  software  was  predicated 
upon  the  preparation  of  a  Unit  Development 
Folder  (UDF)  for  each  unit  of  code  generated. 
The  basic  unit  of  code  on  MET  was  the  Computer 
Program  Module  (CPM) .  The  UDF  served  as  the 
central  historical  source  of  development 
Information  for  a  given  CPM  and  the  basis  from 
which  contract  deliverable  documents  were 
prepared. 
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Figure  1  Met  Computer  System  Configuration 
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Figure  2  Met  Software  Work  Breakdown  Structure 


The  MET  software  design  was  developed 
hierachically  f com  the  top  down  in  accordance 
with  a  software  work  breakdown  structure  of  five 
levels.  These  levels  satisfied  the  objectives 
of  grouping  related  functions  and  identifying 
units  of  code  down  to  the  lowest  level 
controlled.  Figure  2  illustrates  the  structure 
of  MET  software  and  expands  one  of  its  subsystem 
areas. 

Program  modules  and  submodules  were  made  to 
satisfy  stringent  conditions  of  size  (2C0 
FORTRAN  statements  or  less),  function, 
testability  and  reference  to  specific 
requirements  in  the  contract  specification.  All 
were  written  using  FORTRAN-77,  which  includes 
the  structured  programming  language  constructs 
Of  IF,  THEN ,  ELSE,  00  UNTIL,  DO  WHILE  etc., 
which  contributed  to  the  generation  of  clear, 
easy  to  follow  code. 

Software  Design 

The  software  design  was  accomplished  by  the 
preparation  of  five  basic  documents  which 
defined  the  system  performance,  design  ana 
implementation  specifications.  These  documents 
were  utilized  as  the  basis  for  the  conduct  of 
the  Preliminary  and  Critical  design  reviews. 
The  five  documents  in  question  were: 

Program  Performance  Specification  (PPS)  - 
The  PPS  document  was  based  upon  an  analysis  of 
the  contract  specification  and  described  the 
functions  to  be  performed  by  the  system.  This 
description  included  the  equations/algorithms 
which  were  to  be  Implemented  to  simulate  real¬ 
time  functions  and  the  performance 
characteristics/operational  ranges  of  these 
functions.  Data  Input/Output  diagrams  were 
included  to  illustrate  the  flow  of  inputs  to  and 
outputs  from  the  various  system  function 
modules. 

Interface  Design  Speciflcaton  (IDS)  -The 
IDS  document  was  based  upon  the  PPS,  the 
contract  specification  and  a  description  of  the 
system  hardware.  The  IDS  described  in  detail 
the  flow  and  content  of  signals  and  data 
messages  passing  between  the  digital  computer 
processors  which  comprised  the  system. 

Program  Design  Specification  (PPS)  -The 
PDS  document  was  baaed  upon  the  IDS,  which 
defined  the  data  interfaces  between  the  digital 
processors  in  the  system,  and  the  PPS,  which 
specified  all  of  the  functions  to  be  performed 
by  the  system.  The  PDS  provided  a  description 
of  how  the  system  being  designed  would  satisfy 
the  performance  requirements  specified  in  the 
PPS.  Included  was  a  definition  of  the  software 
architecture,  the  identification  and 
description  of  each  oonputer  program  module  to 
be  implemented,  the  Inputs  and  outputs  required 
and  produced  by  each  CPM,  and  diagrams  which 
illustrated  the  flow  of  control  and  data  between 
the  CPMs. 

Program  Description  Document  (FDD)  - 
Preparation  of  the  PDD  was  based  upon  the  three 
predecessor  documents  described  above  and 
included  a  detailed  definition  of  the 
implementation  characteristics  of  each  CPM.  The 


PDD  constituted  a  “code-to"  level  document  and 
included  a  logic  flowchart  of  each  CPM  and  CPSM 
in  the  system,  accompanied  by  a  narrative  keyed 
to  the  flowchart. 

Data  Base  Design  Document  (DBDD)  -  The  DBDD 
was  developed  in  parallel  with  the  PDD  and 
included  a  detailed  definition  and  description 
of  each  data  item  stored  in  the  system  data  base 
and  accessed  by  the  real-time  system  software. 
Included  were  the  definitions  of  variables, 
constants,  tables.  Indexes  and  flags. 

The  unique  characteristic  of  the  MET 
software  design  phase  was  that  a  preliminary 
version  of  each  of  the  above  listed  documents 
was  prepared  prior  to  the  generation  of  a  single 
line  of  application  code.  This  procedure  was 
strictly  enforced  to  maximize  the  thorough 
understanding  of  each  CPM  design  and  to  minimize 
the  number  and  severity  of  problems  which  always 
surface  during  the  software/hardware 
integration  phase. 

Software  Implementation 

Implementation  of  MET  software  was 
accomplished  in  accordance  with  a  standard 
procedure  which  included  five  activities:  Unit- 
Test  Plan,  Code,  Code  Review,  Unit-Testing  and 
Unit~Te3t  Review,  see  Figure  3. 

Unit  Test  Plan  -  The  unit  test-plan 
consisted  of:  a  narrative  which  described  the 
manner  in  which  a  module  was  to  be  tested;  a 
list  of  the  functions  to  be  performed  by  the 
module  in  question  including  appropriate 
references  to  the  contract  specification  items 
being  satisfied;  and  a  series  of  test  cases 
which  had  to  be  executed  in  order  to  demonstrate 
that  each  of  the  functions  were  being  performed 
correctly. 

Code  -  The  code  was  generated  based  upon 
the  flowchart  and  narrative  contained  in  the 
Program  Description  Document  (PDD)  and  with  the 
testing  requirements  of  the  unit-test  plan  in 
mind.  In  practice  the  unit-test  plan  and  coding 
were  generated  concurrently. 

Code  Review  -  A  code  review  was  conducted 
following  code  generation  and  consisted  of  a 
review  of  the  code  logic,  to  ensure  compliance 
with  the  logic  contained  in  the  module  flowchart 
and  a  review  of  the  code  connentary  to  ensure  a 
direct  correlation  with  the  PDD  narrative.  At 
the  time  of  code  review  it  was  a  requirement 
that  all  flowcharts  and  PDD  narratives  had  to  be 
“red-lined*  to  reflect  the  "as  coded*  condition 
of  the  module  in  question. 

Unit  -  Testing  -  Test  of  a  module  was 
accomplished  in  accordance  with  the  unit-test 
plan  for  that  module.  All  of  the  test  cases 
were  executed  and  all  test  results  (console 
outputs,  memory  dumps,  etc)  were  retained  in  the 
unit  development  folder  for  that  module  for 
later  verification. 

Unit-Test  Review  -  A  review  was  performed 
at  the  conclusion  of  module  unit-testing  which 
consisted  of  a  careful  inspection  of  the  unit- 
test  plan,  all  test  cases  (inputs  and 
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UNIT-TEST  PLAN  CODING  CODE  REVIEW 


I'ij'iiro  I:  Software  Implement  ation 


anticipated  outputs)  and  associated  documents, 
(the  PP3,  PDS  and  PDD  were  required  to  be  "red- 
lined"  to  reflect  the  tested  module) . 
Conclusion  of  a  successful  unit-test  review 
resulted  in  the  submission  of  the  module  source 
code  to  the  project  software  library  in 
preparation  for  software/hardware  Integra  tion. 


Conclusions  Regarding  Software  Implementation 


Documentation  Before  Coding  -Software 
implementation  was  accomplished  very  nearly  on 
schedule.  Several  decisions  contributed  to  its 
success,  not  the  least  of  which  wa3  the  decision 
to  document  the  design,  to  the  POD  level,  before 
the  generation  of  application  coding  was  begun. 
The  documented  design  at  the  "code  to"  level 
ensured  that  the  various  system  problems  were 
thought  through  completely  before  coding  was 
started.  As  a  result,  coding  of  the  MBT 
software  was  accorqjlished  much  more  rapidly  than 
planned. 


Weekly  Status  Meetings  -  A  number  of  charts 
and  graphs  were  prepared  to  measure  progress. 
These  were  updated  on  a  weekly  basis  and  used  to 
illustrate  the  status  of  implementation.  Weekly 
status  meetings  were  held  with  all  members  of 
the  software  development  teams  in  attendance. 
Status  was  obtained  by  reviewing  each  item  due 
that  week  and  determining  if  it  had  been 
completed  or  not  and  if  not,  why  not.  If 
necessary  a  new  promise  date  was  established. 
In  this  way,  individual  engineers  were  given  an 
opportunity  to  modify  and  sometimes  establish 
schedule  dates.  This  process  created  an 
environment  wherein  the  meeting  of  schedule 
dates  became  a  matter  of  professional  pride.  In 
addition,  the  meetings  provided  a  needed  forum 
where  problems  could  be  aired  and  resolved  and 
where  management  could  applaud  good  individual 
and  team  performance. 

SOPTWARE/HARDKARE  INTEGRATION 
THE  PROBLBf 


Review  at  Bvery  Step  -  Standard  development 
procedures  were  established  for  program 
implementation  i.e.,  Unit-Test  plan.  Coding, 
Code  Review,  Unit-Testing  and  Unit-Test  Review. 
This  procedure  incorporated  frequent  review  of 
the  implementation  activities  which  contributed 
significantly  to  identifying  problems  early. 


Software/hardware  integration  includes 
those  activities  which  result  in  assembling  all 
of  the  software  components  into  subsystems, 
exercising  the  functions  of  each  subsystem  with 
attendant  hardware,  and  isolating  and  resolving 
interface  and  design  problems.  All  subsystems, 
including  hardware,  are  then  exercised  together 
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producing  a  total  system  operated  under  various 
load  conditions  to  demonstrate  that  all 
functions  for  which  the  system  was  designed 
operated  in  accordance  with  that  design. 

Moment  of  Truth 

It  Is  during  this  process  of  integrating 
software  with  hardware  that  past  sins  of 
superflcal  analysis  and  design  of  both  software 
and  hardware  surface  to  be  seen  by  all.  It 
often  seems  that  the  software/hardware 
integration  process  may  never  end,  that  there 
will  always  be  just  one  more  bug  to  resolve. 

Experience  has  shown  that  the  Integration 
phase  of  any  system  development  is  often 
characterized  by  several  of  the  following  types 
of  problems  with  which  the  reader  may 
emphathize. 

Random  Integration 

Systems  are  comprised  of  many  components, 
both  hardware  and  software.  It  is  not  unusual 
for  the  enthusiasm  for  Integration  to  be 
translated  into  a  frantic  effort  to  leap  forward 
in  assembling  the  various  system  components  in 
order  to  make  grand  and  glorious  system 
Integration  breakthroughs.  Individual  members 
of  the  integration  team  may  have  specific 
knowledge  of  a  particular  system  area  which  they 
may  feel  can  be  integrated  Independently  rather 
quickly,  considering  the  expertise  available  to 
do  the  job.  Some  individuals  are  proponents  of 
top-down  integration  while  others  are  more 
convinced  that  botton-up  is  the  only  way  to  go 
for  3ure,  hence,  if  not  otherwise  directed  each 
brings  his  or  her  particular  bias  to  bear  on  the 
integration  activity. 

It  is  not  uncommon  that  energies,  enthusiam 
and  personal  motivation  of  members  of  the 
integration  team  result  in  several  aggressive 
uncoordinated  thrusts  to  get  the  system  put 
together.  Frequently  these  efforts  conflict 
with  one  another  and  converge  to  a  point  where 
the  process  comes  to  a  grinding  halt  with  no 
further  forward  progress  possible.  At  this 
point  the  integration  teams  must  fall  back, 
regroup  and  resolve  to  do  some  planning. 

Recurring  Familiar  Problem 

The  objective  of  the  integration  activity, 
of  course,  is  to  surface  software  and  hardware 
problems  so  that  they  can  be  resolved.  As  the 
reader  is  probably  aware,  the  integration 
environment  is  usually  a  frantic  one,  with 
personnel  working  multiple  shifts  to  get  the 
system  integrated  within  the  schedule,  which  has 
usually  been  somewhat  compressed  by  this  stage 
of  the  project.  Illusive  problems  are  pursued 
with  3tealth  and  diligence  into  the  late  hours 
of  the  evening  with  notes  being  generated  on 
scraps  of  paper  which  identify  various  anomalies 
surfaced  in  the  course  of  isolating  a  particular 
problem.  Who  has  not  had  the  experience  of 
trouble-shooting  a  problem  which  becomes  more 
familiar  as  it  is  isolated  only  to  recognize 
after  several  minutes  or  hours  that  it  is  a 
problem  which  was  discovered  several  days 


before,  but  forgotten  or  mislaid  In  the 
meantime?  Everyone  involved  resolves  to  Improve 
the  problem  reporting  procedure  so  as  not  to 
repeat  the  experience  of  finding  the  same 
problem  over  again. 

Software/Hardware  Instability 

If  wishes  could  be  granted  during  system 
integration  one  would  opt  to  be  given  an 
opportunity  to  debug  software  on  hardware  known 
to  be  operational  or  to  trouble-shoot  a  hardware 
problem  with  the  same  piece  of  software 
(unmodified)  which  had  surfaced  the  problem. 
Alas,  all  too  frequently  several  perverted 
permutations  of  the  above  conditions  come  to 
pass.  For  example,  a  program  performed  a 
function  successfully  at  10:00  a.m. ,  but  would 
not  work  after  lunch.  Nothing  "really 
significant*  had  occurred  in  the  Interim  except 
that  a  hardware  engineer  swapped  several 
computer  boards  from  one  processor  to  another 
and  removed  a  cable  1  Is  it  ever  possible  to 
keep  the  hardware  base  stable? 

In  another  case  a  problem  with  hardware  had 
been  reported  in  excruciating  detail  by  an 
Integration  programmer.  The  hardware  engineer 
pursued  the  problem  the  following  morning  only 
to  discover  that  he  could  not  reproduce  the 
problem,  thereby  casting  considerable  doubt  on 
the  credibility  of  the  problem  report.  Upon 
closer  inspection  it  was  discovered  that  an 
"unrelated"  modification  had  been  made  to  one  of 
the  programs  used  in  the  test  which  by  chance 
altered  the  test  conditions.  Is  it  ever 
possible  to  keep  the  software  base  stable? 

Inadequate  Planning 

Regardless  of  competence,  dedication  and 
other  sterling  management  qualities,  every 
project  must  suffer  the  occasional  trauma  of 
inadequate  planning,  critical  considerations 
that  simply  went  unnoticed  until  some  manner  of 
crisis  emerged.  Consider  the  case  where  the 
computer  fails  and  the  problem  resolution  is 
isolated  to  a  particular  circuit  board  which  has 
not  been  spared.  That  board,  of  course,  is  the 
type  which  enjoys  a  ninety  day  replacement  time 
and  is  known  by  many  to  have  been  a  high 
failure-rate  part.  Or  the  situation  where  only 
one  engineer  is  an  expert  on  a  particular  aspect 
of  the  system  and  no  one  is  assigned  to  back  him 
up  in  the  event,  heaven  forbid,  he  should 
disappear  from  the  project. 


SOFTWARE/HARDWARE  INTEGRATION  APPROACH 

The  problems  identified  above  are 
undoubtedly  familiar  to  the  reader  and  in  the 
final  analysis,  considering  human  nature  being 
what  it  is,  they  may  be  impossible  to  elminate 
entirely.  Much,  however,  can  be  done  to 
minimize  their  effect.  The  following  paragraphs 
discuss  the  procedures  established  and  used  on 
the  MET  project  and  the  actions  taken  which  in 
ths  author's  view  contributed  in  large  measure 
to  a  reduction  of  the  anguish  associated  with 
these  problems. 
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Integration  Plana  and  Procedures 

In  order  to  reduce  the  more  bizarre  aspects 
of  the  random  Integration  activity  discussed 
above  a  careful  examination  was  made  of  the 
entire  system  and  Its  components.  A  top-down 
approach  was  used  In  viewing  the  system  and  it 
was  decided  that  four  levels  of  testing  were 
required,  if  the  integration  process  was  to  be 
conducted  in  an  orderly  manner,  see  Figure  4. 

CPU  Unit-Test  Plans/Procedures  (Level  4>  - 
The  lowest  level,  (Level  4) ,  Unit-Test  plans  and 
procedures  were  developed  for  each  computer 
program  module  (CPM) .  These  unit  -  test 
plans/procedures,  as  previously  described, 
consisted  of  a  test  plan  narrative,  (which 
described  the  testing  approach),  a  function 
list,  (all  functions  performed  by  the  CPM)  and  a 
series  of  test  case  descriptions  (inputs, 
outputs,  anticipated  results)  see  Figure  S. 
Level  4  testing  was  intended  to  demonstrate  that 
a  particular  CPM  successfully  performed  all  of 
the  functions  for  which  it  was  designed.  In 
this  way,  as  each  CPM  was  entered  into  the 
project  software  library,  following  the 
successful  conclusion  of  the  CPM  unit-test 
review,  a  high  level  of  confidence  was 
established  that  the  individual  CPM  components 
worked. 

Component  Group  Testing  (Level  3)  -Level  3 
testing  really  constituted  the  first  attempt  at 
integrating  software  modules  with  the  equipment 
required  to  support  the  functions  that  the 
modules  performed.  To  ensure  that  firm 
direction  was  established  for  use  by  integration 
team  members  in  assembling  the  various  system 
software  and  hardware  components,  detailed 
plans  and  procedures  were  developed  for  all 
level  3  testing. 

Level  3  integration  testing  consisted  of 
assembling  a  group  of  software  modules  within  a 
subsystem  and  exercising  them  as  a  unit.  Module 
groupings  were  selected  on  the  basis  of  the 
functions  they  performed  i.e.,  related 
functions  which  could  be  tested  independently  of 
other  function  groups. 

Integration  plans  for  level  3  consisted  of: 
a  narrative,  describing  the  approach  used  to 
aggregate  and  teat  the  modules  and  related 
hardware;  a  li3t  of  functions  to  be  demonstrated 
in  the  course  of  integration:  and  a  set  of 
detailed  step-by-step  procedures  to  be  followed 
in  executing  each  functional  action,  see  Figure 
6.  The  function  list  identified  the  function(s) 
to  be  demonstrated,  the  CPMs  which  would  be 
active  during  the  exercise  of  the  function  in 
question,  and  the  hardware  and  other  support 
software  required  to  run  the  test.  The  step-by- 
step  procedures  indicated:  the  action  to  be 
taken  (e.g.,  activate  aircraft  6):  the  device  at 
which  the  action  was  performed  (e.g,,  instructor 
console):  and  the  expected  result  (e.g., 
aircraft  6  indicated  as  active  on  the  vehicle 
selection  tableau) . 

If  the  anticipated  result  was  observed  to 
be  correct,  that  procedure  step  was  checked  off, 
otherwise  a  comment  was  entered  which  described 
the  actual  result  and  what  was  incorrect  about 


it.  As  problems  were  surfaced  each  was 
documented  on  a  System  Problem  Report,  a 
procedure  discussed  in  a  later  paragraph. 

Subsystem  Testing  (Level  2)  -  The  testing 
performed  at  this  level  was  directed  toward  the 
operation  of  a  complete  subsystem,  (i.e.,  Master 
Control,  Passive  Sonar,  Active  Sonar  and  Radar). 
As  can  be  seen  from  Figure  4,  level  2  testing  is 
simply  a  further  integration  of  all  the  module 
groupings  which  together  comprise  a  subsystem. 
Integration  test  plans,  identical  in  format  to 
level  3,  were  generated  which  provided  guidance 
for  conducting  a  step-by-step  demonstration  of 
an  entire  subsystem.  Some  interactions  between 
subsystems  were  exercised  under  conditions  of 
maximum  load  e.g.,  all  aircraft  active  and  being 
tracked,  all  weapons  being  fired  at  the  same 
time,  maximum  number  of  surface  ships  and 
submarines  active,  etc.  Once  again  as  problems 
were  surfaced  they  were  documented  using  the 
System  Problem  Report. 

Lessons  Learned  -  Experience  during  the  MET 
integration  indicated  that  generation  of 
Integration  Test  plans  (ITP)  was  instrumental  in 
forcing  the  required  attention  and  thought  to 
the  questions  of  integration  sequence,  schedule 
requirements  for  various  hardware  components 
and  the  need  for  moving  from  single  to  multiple- 
shift  operations. 

Initially,  level  3  ITPs  were  generated  in 
parallel  with  and  slightly  ahead  of  the 
integration  activities.  There  were  occasions, 
however,  when  working  to  resolve  a  problem  took 
precedence  over  the  continued  preparation  of 
integration  procedures.  When  this  situation 
occurred  a  whiplash  effect  was  felt.  That  is, 
after  the  problem  was  solved,  the  integration 
effort  slowed  down  to  a  near  halt  while  the 
procedures  needed  to  continue  the  integration 
effort  were  generated. 

In  retrospect,  I  am  convinced  that 
preparation  of  the  level  3  integration 
procedures  should  have  taken  place  prior  to  the 
initiation  of  software/hardware  integration. 
Availability  of  these  procedures  would  have 
ensured  a  smoother  integration  activity  and  a 
more  efficient  utilization  of  human  and  computer 
resources.  The  level  2  integration  plans  and 
procedures,  however,  were  completed  prior  to  the 
performance  of  subsystem  testing.  Level  2 
integration  was  accomplished  more  rapidly  and 
without  diversion. 

Integration  Team  Responsibilities 

Computer-driven  training  systems  such  as 
the  MET  are  comprised  of  both  hardware  and 
software,  as  a  result  development  is  most  often 
accomplished  by  groups  of  software  and  hardware 
people  who  work  in  close  cooperation  to  define 
functional  interfac  s  during  the  system 
definition  phase.  Following  thi3  phase  the 
groups  generally  work  independently  to 
implement  the  system  design.  At  a  point  in  the 
schedule  (integration)  the  two  groups  come 
together  to  assemble  both  aspects  of  the  system. 
In  an  integration  environment  where  software 
people  are  responsible  for  software  integrity 
and  hardware  people  for  hardware  integrity  it  is 
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Figure  4:  Hieraichy  of  Integration  Teat  Planning 
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not  unusual  to  find  a  considerable  amount  of 
finger-pointing.  When  a  problem  is  surfaced 
which  cannot  be  easily  ascribed  to  either  area 
the  blaming  of  hardware  by  the  software  people 
and  vice  versa  has  been  known  to  occur,  a 
practice  which  Is  definitely  counter-productive 
to  getting  the  job  done. 


MBT  integration  teams  were  established  for 
each  subsystem  and  contained  both  hardware  and 
software  people.  The  integration  team  leader, 
whether  of  software  or  hardware  persuasion,  was 
given  responsibility  for  all  aspects  of 
subsystem  integration.  This  organization 
tended  to  lessen  any  parochial  feelings  among 
team  members.  A  common  motivation  was 
established  that  all  of  the  problems  surfaced 
were  "team  problems"  and  these  were  dealt  with 
in  a  remarkable  atmosphere  of  comradeship  and 
truly  shared  responsibility. 

Lessons  Learned  -  Granting  the  Integration 
team  leader  full  responsibility  over  both  the 
hardware  and  software  included  in  a  subsystem 
did  provide  him  with  the  flexibilty  to  change 
priorities  of  certain  activities  which 
increased  the  probability  of  remaining  on  the 
integration  schedule. 


Problem  Reporting  t  Resolution 

As  was  previously  stated,  one  of  the 
objectives  of  the  integration  activity  is  to 
surface  problems  and  resolve  them.  Regardless 
of  the  difficulty  of  isolating  a  problem  it 
seem9  as  if  it  is  even  more  of  a  chore  to 
document  it.  Claims  of  "It  will  take  me  less 
time  to  just  fix  it  than  write  it  up..."  or  "I 
won't  forget  this  one,  I'll  fix  it  tomorrow..." 
are  frequently  uttered  to  salve  the  conscience 
of  an  integration  team  member  who  has  decided 
that  he  oc  she  does  not  have  to  document  a 
problem. 


Problem  reporting  during  software/hardware 
integration  is  a  time  when  benevolence  in 
management  Is  a  self-defeating  indulgence.  No 
exceptions  should  bu  tolerated  -  all  problems 
large  or  small  must  be  documented  as  they  are 
encountered  during  integration.  If  this  rule  is 
not  enforced  it.  is  certain  that  problems  will  be 
forgotten  only  to  be  rediscovered  time  and  time 
again  at  great  cost  in  both  time  and 
frustration. 

System  Problem  Report  (SPR)  -  An  SPR  form 
wa3  designed  for  use  on  the  MBT  project  which 
secved  as  a  vehicle  for  documenting  all 
problems,  software  and  hardware.  The  form  was 
used  to  record  a  problem  identification  number 
and  descriptor,  date  encountered,  originator,  a 
brief  description  of  the  symptoms  and  an 
educated  gue3s  as  to  the  cause.  The  SPR  was 
submitted  to  the  software  Quality/Manageraent 
(Q/M)  group  representative  who  maintained  the 
problem  log  and  other  status  charts,  see 
Figure  7. 

An  interesting  attitude  developed  among 
some  members  of  the  integration  teams.  Some 


teams  generated  large  numbers  of  SPRs  while 
others  spawned  but  a  few.  It  was  discovered 
thet  certain  individuals  regarded  the 
generation  of  SPRs  as  an  indication  that  their 
software/hardware  would  be  considered  by 
managment  to  have  been  poorly  designed  and/or 
implemented)  hence,  they  were  not  inclined  to 
generate  SPRs.  It  became  necessary  to  advise 
all  members  of  the  integration  teams  that  the 
only  way  to  measure  progress  was  by  collaring 
the  number  of  SPRs  generated  to  the  number 
resolved. 


If  SPRs  were  being  generated  and  resolved 
and  the  integration  procedures  were  being 
executed  at  a  resonable  rate  this  was 
interpreted  as  progress.  However,  if  few  SPRs 
were  generated  and  the  integration  procedures 
were  not  moving  ahead  very  quickly  this  was 
interpreted  as  no  progress  and  frowned  upon.  It 
took  quite  some  time  to  get  the  SPR  procedure 
accepted  and  used  by  all  team  members,  however, 
once  accepted  it  served  the  integration  activity 
well. 


Software  Change  Report  (SCR)  -  The  SCR  was 
utilized  as  a  vehicle  for  closing  out  an  open 
SPR  which  found  its  resolution  in  a  modification 
to  a  program.  The  SCR  contained  information 
which  described  the  change  being  made  to  the 
program  in  question  and  identified  the  SPR  which 
was  being  resolved  by  the  SCR.  The  SCR  was  also 
used  by  the  project  software  librarian  a3  a 
request  to  perform  an  update  to  the  software 
baseline  maintained  in  the  project  library. 


Figure  7  illustrates  the  procedure 
followed  to  control  SPRs  and  SCRs,  The 
integration  team  surfaced  problems  which  they 
recorded  on  the  SPR  form  which  was  then 
submitted  to  the  software  (Q/M)  representative 
who  maintained  and  published  SPR  log,  SPR  list 
and  SPR  status  reports.  Integration  team 
members  pursued  the  problem  and  devised  a  fix 
which  would  resolve  the  problem.  If  the  fix  was 
implemented  in  software,  the  SCR  was  written  and 
the  source  code  updates  generated  and  submitted 
to  the  software  Q/M  representative  who  tasked 
the  project  librarian  to  perform  an  update  of 
the  library  source  code  for  the  module  being 
changed.  The  modified  module  was  then 
recompiled  and  the  updated  version  placed  on  the 
integration  disc  to  support  continued 
integration  activities. 


Lassons  Learned  -  It  became  apparent  very 
early  in  the  integration  phase  that  given  any 
opportunity,  integration  team  members  would 
often  opt  not  to  document  problems.  It  is 
recommended  that  a  firm  practice,  requiring  that 
all  system  problems  be  documented,  be 
established  and  rigorously  enforced  throughout 
the  integration  period.  The  status  of  SPR 
generation  versus  SCR  production  provides  a 
fairly  accurate  picture  of  integration 
progress.  When  the  rate  of  SCR  generation 
exceeds  SPR  production  it  may  be  fair  to  assume 
that  integration  is  converging  to  a  state  of 
completion. 
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Figure  7:  SPR  &  SCR  Control  Procedure 


Software/Hardware  Configuration  control 

Integration  testing  is  frequently 
described  as  the  checking-out  of  marginal 
software  on  marginal  hardware.  There  may  be 
some  truth  in  such  characterisations.  If  that 
be  the  case,  no  effort  should  be  spared  to 
minimise  the  number  of  variables  in  the 
Integration  environment  which  contribute  so 
much  to  confusion.  The  integrity  of  the 
software  and  hardware  baselines  being  used  by 
the  integration  people  must  be  preserved.  That 
is,  no  change  should  be  made  to  programs  or 
equipments  unless  they  are  strictly  controlled 
and  made  known  to  all  members  of  the  integration 
team. 

Project  Software  Library  -  Control  of  the 
software  baseline  during  integration  was 
accomplished  by  the  establishment  of  a  Project 
Software  Library.  as  modules  successfully 
completed  the  Unit-Test  Review  process  they  were 
entered  into  the  library.  The  software  project 
librarian,  who  was  a  member  of  the  software  Q/H 
group,  was  tasked  with  providing  a  copy  of  the 
controlled  programs  (load  modules)  to  the 
integration  teams.  As  a  result  all  integration 
was  performed  on  a  known  software  baseline. 


As  problems  were  encou  tered  and  resolved, 
integration  personnel  prepared  source  update 
commands  for  each  resolution  and  completed  an 
SCR  which  not  only  described  the  change,  but  was 
also  used  to  close  a  corresponding  SPR.  On  a 
periodic  basis  (sometimes  daily)  the  librarian 
performed  an  update  of  the  programs  In  the 
library,  increased  the  revision  number  of  the 
modules  which  had  bean  changed,  reconciled  these 
modules  and  resubmitted  updated  load  modules  to 
the  integration  teams  for  continued  use.  Ho 
changes  were  accepted  for  library  update  unless 
accompanied  by  an  SCR. 

A  similar  procedure  was  followed  to  control 
the  system  data  base.  On  a  periodic  basis 
(usually  weekly)  the  accumulated  data  base 
changes  were  gathered  up  by  the  librarian  and  a 
data  base  update  performed.  Copies  of  the 
revised  data  base  were  then  made  available  to 
each  of  the  integration  teams  for  use. 
Representatives  of  the  software  Q/M  group 
performed  all  of  the  functions  concerned  with 
the  configuration  control  of  source  code  and 
data  base. 

Hardware  Change  Control  -  The  HEX  trainer 
hardware  configuration  is  comprised  of  eight  SEL 
32/75  computers  with  attendant  peripheral 
devices,  several  cabinets  of  CUBIC  developed 
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Interface  equipments,  sonar  sound  generation 
hardware,  digital  radar  landmass  video 
generation  hardware  and  a  large  amount  of 
government  furnished  shipboard  equipment.  Some 
thirteen  hundred  or  so  cables  are  required  to 
Interconnect  the  various  equipments,  which 
provides  some  Indication  of  tiie  configuration 
complexity. 

The  aoftware/hardware  Integration  activity 
was  initiated  somewhat  prior  to  the  completion 
of  the  hardware  development  effort,  which  meant 
that  software  people  were  testing  programs  on 
available  hardware  while  engineering  personnel 
continued  to  install  and  power-up  other 
equipments.  Many  strange  occurrences  were 
observed  during  this  period  which  required  a 
considerable  effort  to  decipher.  Some  events 
were  never  explained.  Supposedly  tr ivial  tasks 
like  disconnecting  cables  to  affix 
identification  labels  or  swapping  of  like 
circuit  boards  between  cabinets  occasionally 
had  a  disruptive  effect  on  the  integration 
activity.  Cable  identification,  which  should 
have  had  no  effect  on  the  integration  in 
progress,  was  occasionally  accomplished  by 
tugging  on  a  cable  which  might  have  been  tangled 
with  another  cable  which,  when  jerked  to  and 
fro,  tended  to  disturb  system  operstion. 
Likewise  the  swapping  of  boards  often  resulted 
in  moving  a  problem  from  one  place  to  another. 

Several  actions  were  taken  to  stabilize  the 
hardware  configuration.  Hardware  status  boards 
were  posted  with  the  current  condition  of 
various  equipments.  Cabinets  were  sealed  and 
log  books  attached  to  each  cabinet  with  a 
standing  rule  that  euthorization  to  enter  the 
cabinet  had  to  be  obtained  and  if  any  change  at 
all  was  made,  it  had  to  be  entered  in  the  tog 
book  for  that  cabinet.  This  ensured  that  a 
migration  history  of  moving  boards  was 
available.  An  access  list  to  particular 
cabinats  was  maintained  identifying  only  those 
engineers  who  could  be  granted  permission  to 
break  the  seal  and  enter  a  cabinet.  Hardware 
quality  control  tasks  were  scheduled  to  be 
carried  out  during  specific  periods  (usually 
weekends)  to  eliminate  the  conflict  with  on¬ 
going  integration  activities. 

Lessons  Learned  -  The  control  of  the 
softwace  baseline  was  effectively  maintained 
with  the  procedures  described;  howevor,  control 
of  the  data  base  wa3  not  accomplished  as  well  as 
it  could  have  been,  due  to  the  fact  that  the 
data  base  update  tools  available  were  too  slow 
(took  too  long  to  run  on  the  computer) .  This 
situation  caused  data  base  updates  to  be 
performed  on  a  weekly  basis  for  a  time,  whan  in 
reality  it  would  have  been  more  productive  and 
supportive  to  the  integration  activity  to  have 
performed  data  base  updates  more  frequently. 

As  regards  tie  control  of  the  hardware 
baseline,  the  overlapping  of  the  integration 
activity  with  that  of  equipment  installation  is 
not  especially  productive  and  is,  in  fact, 
counter  to  one  of  the  basic  axioms  of 
integration,  namely,  minimization  of  the  number 
of  variables  during  testing.  All  equipment 
3houid  be  installed  and  verified  to  be 
operational,  to  the  maximum  extent  possible, 


prior  to  the  initiation  of  software /hardware 
lntegraton. 

Integration  Status  Monitoring 

The  integration  activity  is  by  nature  an 
effort  which  suffers  advances  and  reverses. 
Because  of  its  proximity  to  the  end  of  the 
contract  and  the  fact  that  it  is  a  culmination 
of  the  design  end  implementation  phases,  it 
enjoys  a  wide  management  interest.  Providing 
management  with  a  tangible  assessment  of 
integration  status  is  a  delicate  undertaking,  to 
say  the  least.  In  fact,  it  is  rather  difficult 
to  predict  the  number  of  problems  which  will  be 
encountered  in  the  course  of  integration,  or  how 
long  it  will  take  to  isolate  any  given  problem. 
Several  techniques,  which  are  described  below, 
were  utlilited  to  monitor  the  status  of  the  MET 
integration. 

Configuration  Control  Board  (CCB)  -  A 
configuration  control  board  was  established 
which  consisted  of  software,  hardware  and 
management  personnel.  Participating  on  the  CCB 
were  the  program  manager,  engineering  manager, 
software  manager,  integration  team  leaders  and 
the  software  Q/M  representative.  Other 
individuals  were  requested  to  attend  as  their 
contributions  warranted.  The  CCS  was  convened 
at  laast  once  a  week  and  an  occasion  several 
times  a  week,  depending  upon  integration 
progress  or  lack  thereof. 

The  CCB  reviewed  integration  progress 
against  the  plan,  and  the  progress  made  in  the 
closing  of  SPRs.  Specific  problems  encountered 
during  the  week  were  discussed  and  action  items 
allocated  to  various  individuals.  CCB  meetings 
were  chaired  by  the  software  Q/>l  represenative 
who  was  also  responsible  for  the  preparation  of 
meeting  minutes  and  SPR  reports.  The  board 
successfully  dealt  with  the  prioritizing  of  many 
activities  which  contributed  significantly 
toward  keeping  the  integration  activity 
reasonably  on  schedule. 

Integration  Activity  Charts  -  Three 
principal  mechanisms  were  utilized  to  assist  in 
tracking  integration  progress;  integration 
activity  charts,  PBRT-type  graphic  charts  and 
SPR/SCR  generation  curves.  An  integration 
activity  PERT-type  chart  was  prepared  which 
identified  the  integration  activities  which  had 
to  be  performed  before  the  system  could  be 
considered  ready  for  acceptance  testing.  This 
chart  graphically  portrayed  activites,  duration 
and  time  sequence  and  was  updated  regularly. 
Updating  was  accomplished  by  Indicating  percent 
completion  of  an  activity  using  a  color  code  of 
BLACK  to  indicate  effort  accomplished,  GREEN  to 
indicate  effort  accomplished  ahead  of  schedule 
and  RED  to  indicate  effort  which  had  not  been 
accomplished  and  was  behind  schedule. 

An  integration  activity  chart  was  prepared 
in  columnar  form,  a9  shown  in  Figure  8,  which 
listed  all  of  the  integration  activities  along 
with  their  scheduled  completion  date  and  space 
to  record  the  actual  completion  date.  This 
chart  was  heavily  used  by  the  CCB  during  each 
integration  status  meeting.  Activities  on  this 
chart  were  highlighted  (in  color)  as  they  were 
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Figure  8:  Integration  Activity  Chart 
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Figure  9:  SPR/SCR  Generation  Graph 
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collated.  No  percentage  activity  completion 
was  reflected  on  this  chart,  hence,  it  provided 
a  conservative  view  of  integration  status. 

Graphs  of  the  number  of  SPRs  and  SCRa 
generated,  plotted  as  a  function  of  time, 
provided  an  insight  into  the  progress  being 
made.  The  general  slope  of  the  SCR  curve 
indicated  whether  or  not  the  integration  teams 
were  successfully  closing  problems  out  and 
provided  an  approximation  (by  extending  the 
curve)  of  when  all  problems  would  be  resolved 
and  Integration  completed  (see  Figure  9) . 

Lessons  Learned  -  The  techniques  utilised 
to  monitor  the  status  of  integration  have  proven 
very  useful  in  indicating  not  only  where  the 
integration  activity  was  but  also  how  it  was 
likely  to  proceed.  The  accuracy  of  presentation 
and  utility  of  the  techniques  is  directly 
proportional  to  the  amount  of  energy  applied  to 
the  identification  of  the  integration 
activities.  If  the  Integration  activity  chart 
is  accurate  and  truly  represents  all  of  the 
tasks  which  are  required  to  integrate  the 
system,  the  chart  will  be  invaluable  in 
assessing  status. 

The  SPR/SCR  graph  was  felt  to  be  a  good 
indicator  of  overall  integration  status  when 
considered  in  light  of  the  activity  chart,  if 
the  activity  chart  indicated  a  low  activity 
completion  and  the  SPR/SCR  graph  showed  low  SPR 
generation  it  was  a  clear  indication  that  the 
integration  was  staggering  and  required  review 
and  assistance.  On  the  other  hand  if  the 
activity  chart  indicated  a  reasonable 
completion  of  taks  and  the  SPR/SCR  graph  showed 
a  brisk  SPR/SCR  generation  one  could  conclude 
that  integration  was  progressing  without 
serious  Impediment. 

SUMMARY 

Software/hardware  integration  is  an 
activity  which  is  eagerly  approached  by  all 
members  of  the  system  devlopment  team  because  it 
embodies  the  sweet  challenge  of  making  the  whole 
system  work.  As  exhilarating  as  integration  is, 
it  i3  nevertheless  an  activity  which  is 
frequently  besieged  by  all  manner  of  pitfalls, 
each  of  which  contrives  to  extend  the  schedule 
and  complicate  the  job  to  be  done.  Successful 
integration  i3  certainly  enhanced  by  orderly  and 
thorough  software  and  hardware  design  and 
Implementation.  Without  a  doubt  superficial 
analysis  and  design  of  system  software  and 
hardware  components  will  surface  to  haunt  the 
integration  activity.  Valuable  time  and  effort 
will  be  expended  going  back  to  effect  redesign 
and  modifications. 

Considerable  emphasis  was  placed  on  a 
thorough  design  of  MET  software.  In  fact,  the 
detailed  software  design  was  completely 
documented  before  the  coding  activity  was 
started.  Software  implementation  was  heavily 
monitored  by  the  conduct  of  design  reviews,  code 
reviews  and  finally,  unit-test  reviews.  All  of 
these  activities  ,  I  believe,  contributed  to 
increasing  the  confidence  that  each  software 
component  had  been  thoroughly  tested  before 
being  made  available  for  system  Integration, 


Similar  attention  must  be  given  to  the  design, 
fabrication,  installation  and  test  of  all 
hardware  components. 

Even  after  all  reasonable  precautions  have 
been  taken  to  reach  the  point  of  system 
integration,  a  formidable  task  remains  to  be 
contended  with,  one  which  is  frequently  attended 
by  problems  of  planning  and  configuration 
management  and  problem  monitoring.  These 
problems  were  alleviated  on  the  MET  project  byt 
the  development  of  detailed  integration  plans 
for  the  various  levels  of  the  system 
architectures  the  organization  of  integration 
teams  possessing  both  software  and  hardware 
skills  and  the  authority  over  all  aspects  of  a 
subsystem)  and  the  establishment  of  firm 
procedures  to  control  not  only  problem  reporting 
and  evaluation,  but  also  the  software  and 
hardware  baselines. 

The  MET  software/hardware  Integration 
effort  was  significantly  enhanced  by  the  active 
participation  of  customer  personnel  from  the 
Naval  Training  Equipment  Center  (NTEC)  who  were 
present  as  observers  during  much  of  the  level  2 
and  3  integration  testing.  Their  participation 
resulted  in  the  generation  of  meaningful 
commentary  as  the  system  was  assembled  and 
provided  a  basis  of  common  knowledge  which  was 
taken  into  the  formal  system  acceptance 
activity. 
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ABSTRACT 

Short  schedules  and  changing  requirements  are  common  problems  encountered  when  software 
is  being  developed  for  training  systems.  This  paper  explores  techniques  used  by  Technology 
Service  Corporation  (TSC)  to  overcome  or  avert  such  problems  while  developing  the  B-52  OAS 
Part  Task  Trainer  for  the  Training  Services  Division,  Keesler  Air  Force  Base.  Techniques 
for  dealing  with  limited  resources  (time  and  budget)  include  carefully  exploring,  and  assign¬ 
ing  priorities  to,  system  capabilities  to  determine  the  more  important  requirements;  and 
employing  a  top-down  approach.  Planning  for  changing  requirements  calls  for  identifying  capa¬ 
bilities  that  may  change;  constructing  a  well -documented  software  design  with  application- 
oriented  modularity;  and  scheduling  a  design  freeze,  with  late  requirement  changes  incorpora¬ 
ted  after  completion.  The  paper  presents  step-by-step  descriptions  of  each  technique  and 
provides  examples  relating  directly  to  the  part  task  trainer. 


INTRODUCTION 

The  U.S.  Air  Force  B-52  bomber  is  receiving  a 
major  modification  in  the  form  of  the  Offensive 
Avionics  System  (OAS),  scheduled  for  deployment  in 
1981.  Used  by  the  B-52  radar  navigator  to  perform 
navigational  and  offensive  weapon  delivery  tasks, 
the  OAS  replaces  older  ASQ-38  equipment  and  auto¬ 
mates  some  previously  manual  functions. 

A  weapon  system  trainer  (WST)  is  being  built 
for  the  B-52  as  a  part  of  the  major  weapon  system 
modification.  The  WST  is  scheduled  to  be  com¬ 
pleted  in  the  1983-1986  timeframe,  which  leaves  a 
gap  between  deployment  of  the  OAS  and  availability 
of  the  WST.  Crews  using  the  first  OAS-modified 
B-52  will  need  a  trainer  in  the  interim.  To  meet 
this  need,  TSC,  in  conjunction  with  the  Training 
Services  Division,  Keesler  Air  Force  Base,  has  been 
contracted  to  design  the  software  and  configure 
four  E-5C  OAS  interim  trainers  to  be  used  by  the 
Strategic  Air  Command  (SAC)  in  their  OAS  conversion 
training  program. 

In  approaching  this  assignment,  TSC  recognized 
that,  in  developing  software  for  the  interim 
trainer,  two  major  problems  also  found  in  other 
software  projects  would  be  encountered:  limited 
resources  (short  schedules  and  budgets)  and 
changing  requirements. 

Short  schedules  are  a  function  of  the  relation 
between  the  training  device  and  the  actual  system. 

A  training  device  cannot  be  specified  until  the 
actual  system  is  designed,  but  it  is  needed  as  soon 
as  the  actual  system--either  a  new  one  or  a  modifi¬ 
cation  to  the  existing  one--is  operational,  if  not 
before  then. 

Meeting  the  schedule  and  cost  constraints  is 
then  complicated  by  deciding  which  functions  can 
reasonably  be  provided  by  the  training  system.  The 
first  choice  of  the  users  may  be  a  trainer  that 
gi''es  a  completely  realistic  simulation  of  the 
device.  It  is,  after  all,  difficult  to  identify 
those  skills  that  can  be  effectively  trained  with 
other  methods,  such  as  classroom  instruction,  low- 
cost  training  aids,  or  the  system  itself.  The  cost 
of  such  a  trainer,  however,  may  be  beyond  the 
available  time  and  budget. 


The  second  problem, requirement  changes  during 
development,  commonly  occurs  because  both  the 
trainer  and  the  system  are  being  developed  at  the 
same  time.  Any  changes  to  the  system  under  develop¬ 
ment  must  be  reflected  in  the  trainer;  and,  by 
extension,  since  modifications  to  the  system  are 
likely,  these  same  modifications  must  be  supported 
by  the  training  device. 

This  paper  presents  techniques  that  TSC  has 
found  to  be  effective  in  minimizing,  and  sometimes 
averting,  the  impact  of  these  problems.  These 
techniqui  ;  are  now  being  used  to  develop  the  B-52 
OAS  Part  Task  Trainer  (PTT) .  Before  detailing  them, 
we  describe  the  trainer  as  a  point  of  reference. 


DESCRIPTION  OF  THE  PART  TASK  TRAINER 

The  interim  trainer  is  a  part  task  trainer, 
addressing  some  of  the  tasks,  procedures,  and 
conditions  the  crewmembers  must  handle  with  the 
OAS.  The  crews  receiving  training  will  already  be 
skilled  B-52  navigators.  The  focus  of  the  PTT  will 
therefore  be  on  procedures  training;  specifically, 
those  procedures  unique  to  the  new  OAS.  For 
example,  the  PTT  must  respond  exactly  as  the  OAS 
would  to  all  commands  (button  pushes,  switch 
activations,  etc.),  but  only  those  features  used 
directly  in  navigational  and  weapon-targeting 
procedures  need  to  be  displayed  in  the  simulated 
radar  video. 

The  PTT  is  laid  out  physically  to  provide 
stations  for  the  two  crewmembers  required  to  operate 
the  OAS,  as  well  as  a  position  for  an  instructor 
(Figure  1).  Included  in  this  layout  is  a  mockup  of 
the  OAS  crewstation,  comprising  16  operational 
panels,  four  monochromatic  display  monitors,  and  two 
trackballs.  The  instructor's  position  is  equipped 
with  a  CRT  console  for  setting  up  and  monitoring  the 
training  sessions.  The  hardware  configured  for  the 
part  task  trainer  is  composed  of  the  five  indepen¬ 
dent  subsystems  depicted  in  Figure  2. 

Before  we  describe  the  software  and  the 
approaches  to  lessening  the  impact  of  major 
problems  in  developing  it,  an  overview  of  what  is 
actually  simulated  by  the  software  is  necessary. 
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Figure  2.  OAS  Part  Task  Trainer  Block  Diagram 

281 


i 

t 

i 


The  OAS  will  automatically  navigate  the  B-52 
according  to  a  predefined  flight  plan.  The  crew¬ 
members  will  be  able  to  monitor,  update,  and 
override  this  automatic  navigation  system,  using 
the  OAS  equipment  and  a  variety  of  navigational 
procedures  supported  by  the  PTT.  The  trainer  will 
also  support  OAS  procedures  to  perform  In-flight 
refueling;  preparation  and  delivery  of  Air-Launched 
Ctulse  Missiles  (ALCMs),  Short  Range  Attack 
Missiles  (SRAMs),  and  conventional  bombs;  and 
backup  procedures  used  when  certain  OAS  failures 
occur. 

As  envisioned  in  the  PTT,  the  Instructor  will 
be  able  to  select  a  flight  plan  from  a  set  of 
canned  scenarios  or  modify  any  canned  scenario  to 
generate  a  new  flight  plan.  The  canned  scenarios 
are  representative  of  actual  training  missions, 
and  scenarios  of  up  to  five  hours  of  flight  time 
can  be  generated  within  a  region  covering  the 
Western  United  States.  Sufficient  terrain  features, 
navigational  fixpoints,  and  target  areas  are 
included  within  this  region  to  define  at  least  100 
different  realistic  scenarios.  Navigational  charts 
are  used  to  generate  synthetic  radar  Imagery  for 
display  of  these  features,  fixpoints,  and  target 
areas. 

The  instructor  will  also  have  the  capability 
to  inject  faults  and  malfunctions  at  any  time 
during  the  training  session.  With  the  ability  to 
steer  the  aircraft  off  the  flight  plan  and  to 
accept  corrections  from  the  OAS  crewmembers  for 
recovery,  the  instructor  will  be  acting  as  the  B-52 
pilot.  He  will  also  monitor  crewmembers'  actions 
and  will  be  able  to  freeze  the  training  session  at 
any  time,  give  additional  instruction,  and  then 
resume  the  session. 

As  this  brief  description  indicates,  the  PTT 
is  a  complex  training  device.  This  complexity  can 
be  further  increased  when  the  development  of  soft¬ 
ware  for  such  a  device  is  constrained  by  brief 
schedules  and  changing  requirements.  In  the 
following  sections,  we  present  a  collection  of 
methods  that  vie  have  found  beneficial  in  a  number 
of  applications.  We  first  discuss  methods  for 
reducing  and  overcoming  schedule  problems,  and  then 
techniques  that  facilitate  incorporation  of 
requirement  changes. 


SOMF  SOLUTIONS  TO  TIME  AND  BUDGET  PROBLEMS 

As  mentioned  in  the  Introduction,  completion 
of  the  training  device  must  coincide  with  or 
precede  that  of  the  weapon  system,  often  resulting 
in  short  schedules.  In  addition,  schedule  slips 
may  occur,  owing  to  Inaccurate  estimates  and  the 
vicissitudes  of  daily  life:  illness,  employee 
turnover,  machine  failures,  and  requirement  changes. 
Specific  methods  TSC  uses  to  anticipate  potential 
schedule  problems  and  to  avoid  cost  overruns 
Include  carefully  exploring,  and  assigning  priori¬ 
ties  to,  system  capabilities  to  determine  the  most 
important  training  functions;  and  employing  a  top- 
down  approach  so  that  a  very  limited,  skeleton 
system  that  performs  some  of  the  required  functions 
is  developed  first,  followed  by  versions  that 
successively  add  more  capabilities  until  the  system 
Is  complete.  These  two  techniques  are  actually 
interconnecting,  with  assigning  of  priorities  to 
different  functions  contributing  to  the  definition 
of  development  stages  or  versions  in  the  top-down 
approach.  Careful  choice  of  programming  language 


and  computer  system,  and  Incorporation  of  a  pro¬ 
gramming  design  language  also  contribute  to 
averting  schedule  problems  and  are  discussed  at 
the  end  of  this  sect4on. 

Assigning  Priority 

The  system  specification  usually  details  the 
functional  requirements;  these  functions,  however, 
can  be  further  ranked  by  the  user  Into  such 
categories  as: 

1.  Necessary  for  training. 

2.  Important  function  without  which  the 
system  will  be  marginally  useful. 

3.  Important  function  definitely  desired 
by  the  user  but  which  could  be  taught, 
if  need  be,  using  another  medium. 

4.  Nonessential  or  desirable  function. 

The  user's  initial  ranking  of  system  functions 
may  place  all  functions  In  Category  1,  and 
convincing  the  user  of  the  Importance  of  ranking 
them  may  be  difficult  (or  even  Impossible)  until 
schedule  slips  occur.  It  Is  hoped,  however,  that 
discussions  with  the  user  will  be  fruitful  for 
identifying  the  more  critical  training  functions. 
This  ranking  can  then  be  used  in  top-down  develop¬ 
ment  as  part  of  the  process  of  identifying  the 
successive  versions. 

Top-Down  Development 

In  top-down  software  development,  described  by 
Yourdon,  (1)  the  high-level  design  of  the  system 
is  followed  by  implementation  of  a  barebones 
system  that  performs  some  of  the  required  func¬ 
tions,  followed  by  versions  that  successively  add 
more  capabilities  until  the  system  Is  complete. 

By  contrast,  in  the  classicial,  bottom-up 
approach,  the  entire  system  is  designed,  coded, 
debugged,  and  then  Integrated.  If  a  scheduling 
problem  occurs,  the  developer  and  user  may  find, 
when  the  deadline  arrives,  that  although  100 
percent  of  the  code  is  written,  nothing  works. 

The  same  schedule  slip  with  the  top-down  approach 
will  find  a  working  version  that  may  provide  75 
percent  of  the  total  system  functions.  And 
although  the  user  will  not  be  satisfied  with  75 
percent  of  a  system,  that  75  percent  will  be  more 
acceptable  than  50,000  lines  of  code  that  are 
useless  because  they  have  not  been  integrated. 

Furthermore,  a  user  is  more  likely  to  have 
confidence  in  your  ability  to  finish  the  job  if 
he  can  see  a  working  version  of  the  partial  system. 
For  example.  Version  1  of  the  PTT,  described 
below,  was  valuable  because  we  were  able  to 
demonstrate  it  to  the  customer,  which  was  more 
effective  than  telling  him  10,000  lines  of  code 
had  been  written.  And,  if  the  versions  are  care¬ 
fully  defined,  the  customer  may  be  able  to  start 
training  with  the  current  version  while  the 
training  device  is  being  completed. 

The  most  important  functions,  decided  by  the 
priority  ranking  discussed  above,  are  scheduled 
for  Implementation  In  the  early  versions,  the  less 
Important,  in  the  later.  This  scheduling  of 
functions  in  versions  is  a  compromise  between  the 
logical  steps  required  to  develop  the  software  and 
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the  desire  to  provide  intermediate  working  versions 
that  are  useful  for  training.  For  Instance,  the 
radar  position  fix  procedure  may  be  the  most 
Important  training  function;  however,  since  It 
requires  panel  Inputs,  radar  graphics,  trackball 
control  of  the  crosshair,  simulation  of  aircraft 
flight  and  navigation  systems,  etc.,  Intermediate 
versions  are  defined  to  develop  the  basic  building 
blocks,  and  then  the  position  fix  procedure  Is 
scheduled  for  Implementation  in  the  next  version, 

On  the  PU  project,  the  users  (SAC)  were  very 
cooperative  In  ranking  the  priority  of  training 
funclons.  From  this  ranking,  we  defined  seven 
versions  (Table  1)  for  the  top-down  development  of 
the  PTT.  The  first  three  versions  provide  a 
logical  development  of  the  basic  functions  of  the 
system  required  to  support  the  operational  pro¬ 
cedures  scheduled  in  Versions  4  through  7.  Versions 
1  through  4  have  been  Implemented,  and  Version  5  Is 
under  way.  The  estimated  time  to  complete  each 
version  ranges  from  four  to  nine  weeks. 

Version  1  may  appear  so  basic  as  to  be  trivial. 
Its  completion,  however,  was  a  significant  event 


because,  for  one,  this  version  required  that  two  of 
the  four  system  Interfaces  work:  Input  received 
from  the  Instructor's  console,  and  output  generated 
to  the  display  subsystem.  Yourdon  discusses  In 
detail  the  advantages  of  top-down  development  In 
testing  major  interfaces  early  In  the  development 
cycle. (1)  Often  these  Interfaces  are  where  prob¬ 
lems  occur,  and  such  problems  are  usually  the  most 
difficult  to  correct.  Second,  In  this  first 
version,  although  only  three  of  12  tasks  are 
implemented  to  any  degree,  these  three  tasks  are 
scheduled  and  communicate  with  each  other,  exer¬ 
cising  a  majority  of  the  system  executive  routines 
that  handle  task  interfacing,  another  area  where 
significant  problems  often  occur. 

Version  2  added  considerably  more  capabilities: 
all  subsystem  Interfaces  were  exercised;  Inputs 
from  one  switch  panel  were  processed  by  the  soft¬ 
ware,  allowing  evaluation  of  response  times;  and 
most  alphanumeric  display  formats  and  static  radar 
video  were  displayed.  Version  4  should  be  adequate 
for  training  the  first  B-52  crews,  because  ALCM 
and  SRAM  procedures  training  Is  not  required  for 
them;  and  Version  5  is  expected  to  provide  90 
percent  of  the  necessary  training  functions. 


Version  1 


TABLE  1.  OAS  PART  TASK  TRAINER  VERSIONS 
Version  5 


Instructor  starts  system  in  Run  mode 
Aircraft  flies  in  a  straight  line 
Display  prime  mission  data,  left- hand- side 
data 

Version  2 

Instructor  maneuver  aircraft  commands 
Aircraft  flies  default  scenario 
Crewmember  inputs  from  Integrated  Key 
Boards  (I KBs): 

Select  MFD 
Select  Format 
Select  Menu 
FLY  TO  Command 
Display: 

Static  radar  video 
Static  alphanumeric  data 

Version  3 

Instructor  select/preview  command 
Instructor's  real-time  parameters  display 
Navigation  errors  modeled 
Crewnember  inputs: 

Remaining  IKB  functions 
Radar  Navigator's  Management  Panel 
Display: 

Dynamic  radar  video  (default  format) 
Dynamic  alphanumeric  data 

Version  4 

Procedures: 

Bomb  run 

Auto  fixpomt  sequencing 
Crewmember  inputs: 

Bomb  panels 

Special  weapons  panels 
Display: 

Crosshair  and  residuals 
All  radar  video  formats 


Procedures: 

ALCM  weapon  procedures 
SRAM  weapon  procedures 
High  altitude  calibration 
Radar  position  fix 
OAS  initialization 
Crewmember  inputs: 

Weapons  Control  Panel 

Version  6 

Instructor  commands: 

Fault 

Wind 

Alternate  nav  heading  error 
Missile  all/none  status 
Freeze/ resume 
Procedures: 

Alternate  true  heading  calibration 
OAS  bus  failure 
Panel  failures 
Crewmember  inputs: 

Remaining'  panels 

Version  7 

Instructor  capabilities: 

Edit/save  scenario 
Post- run  mode 
Procedures: 

Point  parallel  rendezvous 
Alternate  bomF  run 
Terrain  correlation  fix 
Low  altitude  calibration 
Display  altitude  ribbon 
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To  show  how  the  top-down  development  relates 
to  the  actual  code  written.  Table  2  tabulates  the 
estimated  effort  to  complete  each  task  and  library 
of  routines  for  each  version.  It  shows  that,  In 
Version  1,  a  major  portion  of  code  In  the  library 
routines  for  error  handling,  data  passing,  Input, 
output,  etc.,  was  developed  and  exercised  to  sup¬ 
port  a  very  small  amount  of  task  code.  By  Version 
2,  nearly  all  library  routines  were  operational. 

This  Information  is  also  beneficial  In  explaining 
to  the  customer  how  much  effort  goes  into  a 
Version  1  to  produce  what  may  be  a  small  subset  of 
visible  functional  capabilities. 

TSC  has  found  that,  using  top-down  development, 
the  high-level  design  for  the  system  should  be 
completed  before  any  versions  can  be  implemented. 
That  Is,  the  system  executive  routines  which  per¬ 
form  such  functions  as  Input/output,  data  base 
control,  and  Intertask  communication  must  be 
defined,  as  well  as  each  task  and  data  base.  These 
definitions  should  include  the  functions  of  each 
entity,  and  all  inputs  and  outputs  at  the  func¬ 
tional  level.  For  example,  the  task  that  provides 
the  instructor's  display  should  Identify  the 
current  aircraft  parameters  of  speed,  heading,  and 
altitude  as  inputs  from  a  named  data  base,  but 
would  not  have  to  specify  the  format  of  the  data 
base.  Given  this  high-level  design,  Implementation 
of  each  version  can  proceed  and  the  developers  can 
be  assured  tnat  no  major  problems  will  be  dis¬ 
covered  in  Version  5  that  could,  for  instance, 
necessitate  a  redesign  of  code  developed  for 
Version  1 . 

Top-down  development  also  makes  estimating 
easier,  as  well  as  having  other  advantages.  The 
software  staff  prefers  it  because,  instead  of  one 
long  cycle  of  design,  coding,  and  integration,  the 
project  is  segmented  Into  shorter  cycles  of  design, 
coding  and  Integration  for  each  version.  The 
completion  of  each  version  results  In  the  software 


team  having  a  feeling  of  accomplishment  and  enthusi¬ 
asm  to  tackle  the  next  version.  Having  several 
such  cycles  makes  estimating  the  time  to  complete 
the  remaining  versions  easier.  By  contrast.  In 
the  bottom-up  approach,  knowing  how  long  it  took  to 
design  and  write  all  the  code  does  not  help  to 
estimate  how  long  It  will  take  to  Integrate. 

Programing  LanguaQe  and  Computer  System  Choice 

Short  schedules  also  encourage  careful  choice 
of  programming  language  and  computer  system  and 
support  software.  Coding  time  Is  significantly 
reduced  when  high-level  languages  are  used  Instead 
of  assembly  language.  Any  Inefficiencies  In 
program  size  caused  by  the  high-level  language  can 
usually  be  offset  by  purchasing  more  memory.  As 
for  inefficiencies  in  execution  time  caused  by  the 
high-level  language,  those  portions  of  code 
detected  as  causing  timing  problems  can  be  rewrit¬ 
ten  in  assembly  language. 

Using  a  language  with  which  the  programmers 
are  already  familiar  also  helps.  In  the  B-52  OAS 
PTT,  the  IFTRAN  language  was  used.  IFTRAN  Is  a 
structured  FORTRAN  language  used  by  TSC  for  nearly 
all  programming  projects. 

The  minicomputer  selected  for  the  PTT  is 
compatible  with  TSC's  in-house  system;  therefore, 
the  PTT  project  was  able  to  draw  upon  a  pool  of 
programmers  experienced  with  the  system  and  a 
library  of  routines  and  software  tools.  The 
vendor-supplied  operating  system  also  provides 
many  of  the  capabilities  required  for  real-time 
systems:  multitasking,  semaphores,  priority 
levels,  mapped  I/O,  etc.,  thus  minimizing  the 
number  of  executive  routines  to  be  generated.  The 
vendor-supplied  software  includes  capabilities  to 
facilitate  software  development:  timesharing  with 
virtual  memory  management  that  allows  several 
programmers  to  develop  and  test  code  simultaneously. 


TABLE  2.  PERCENTAGE  OF  SOFTWARE  DEVELOPED  FOR 
EACH  MAJOR  MODULE, BY  VERSION 


VERSION 

|_J_ 

2 

3 

4 

6 

6 

i  7 

TASKS: 

COM 

5 

30 

40 

50 

60 

70 

100 

I  NO 

70 

70 

70 

70 

100 

CRW 

80 

90 

90 

100 

100 

100 

RIK 

40 

50 

60 

70 

80 

100 

NIK 

40 

50 

60 

70 

80 

100 

RNM 

20 

40 

80 

90 

100 

WPN 

15 

70 

90 

100 

AIR 

5 

30 

40 

80 

90 

100 

100 

MFD 

20 

50 

60 

80 

90 

100 

PMD 

30 

40 

40 

80 

90 

90 

100 

XHR 

5 

50 

80 

90 

100 

RDR 

30 

40 

75 

90 

100 

100 

LIBRARIES: 

10  LIB 

30 

70 

90 

100 

100 

100 

100 

EXEC  LIB 

50 

90 

100 

100 

100 

100 

100 

DB  LIB 

80 

100 

100 

100 

100 

100 

100 

Q  LIB 

100 

100 

100 

100 

100 

100 

NAV  LIB 

80 

90 

100 

100 

100 

100 

100 

ERR  LIB 

60 

75 

75 

100 

100 

100 

100 
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and  a  source- level  debugger  that  allows  programners 
to  debug  on-line  In  the  high-level  language. 

Another  suggestion  for  speeding  up  progress  In 
software  development  Is  to  have  project  management 
Intervene  when  design  discussions  drag  on.  When 
the  programming  staff  is  undecided  over  alternative 
methods  and  no  outstanding  risks  are  identified, 
the  chief  designer  must  pick  one  method  and  con¬ 
tinue.  The  method  that  is  most  straightforward  to 
implement  should  be  the  one  selected, 

Program  Design  Language 

Finally,  it  is  important  that  development 
standards  not  be  abandoned  because  the  schedule  is 
short.  For  instance,  at  TSC,  the  first  step  in 
software  development  is  to  express  the  design  in 
PDL  (program  design  language).  POL  is  an  English 
language  description  of  the  design  with  a  few 
structured  programming  keywords  such  as  IF,  ORIF, 
ELSE,  REPEAT,  WHILE.  Expressed  in  PDL,  the  design 
is  structured,  machine- independent,  and  understand¬ 
able  by  nonprogrammers,  such  as  the  user.  Figure  3 
is  a  sample  PDL  listing  of  a  routine  that  controls 
cursor  movement  on  a  menu-driven  display. 

The  entire  software  team  reviews  the  PDL  to 
ensure  that  the  PDL  is  understandable  by  everyone, 
to  detect  errors  and  omissions,  and  to  suggest 
improvements  in  the  design.  Once  the  design  is 
approved,  code  is  generated  and  also  reviewed. 

Code  is  inserted  in  the  same  source  file  with  the 
PDL,  and  preprocessors  allow  the  listing  of  PDL 
only,  code  only,  or  code  with  PDL  inserted  as 
comments . 

Use  of  PDL  and  reviews  significantly  shortens 
the  integration  and  documentation  phases  of  the 
software  process  in  the  following  ways.  First, 

PDL  design  reviews  eliminate  many  of  the  errors 
that  are  normally  not  detected  until  software  inte 
gration.  Second,  reviews  of  PDL  and  code  ensure 
that  each  person  knows  enough  about  all  the  soft¬ 
ware  to  detect  and  correct  many  errors  quickly 
without  involving  other  team  members.  Third, 
software  debugging  is  easier  with  PDL  embedded  as 
explanatory  comments  in  the  code.  Fourth,  keeping 
PDL  and  code  together  ensures  that  coding  changes 
are  also  reflected  in  the  PDL.  Thus,  at  project 
completion,  PDL  listings  can  be  used  as  final 
program  documentation.  Finally,  the  requirement 
that  the  software  design  be  expressed  in  PDL 
ensures  that  the  design  is  documented--and  not  in  a 
disorderly  set  of  notes  or  stored  in  a  programmer' s 
head.  Thus,  if  the  programmer  becomes  ill  or 
leaves  the  project,  the  disruption  is  minimized 
because  another  prograumer  can  get  "up  to  speed" 
more  easily. 


SOME  SOLUTIONS  TO  THE  PR08LEM 
OF  CHANGING  REQUIREMENTS 

Requirement  changes  during  development  of  a 
trainer'  device  are  unavoidable  when  the  system 
simulated  by  the  trainer  is  undergoing  simultaneous 
development  or  modification.  In  such  a  situation, 
a  freeze  should  be  invoked  on  the  trainer  design 
so  development  can  proceed  without  upheavals  and 
delays  caused  by  changing  specifications.  Follow¬ 
ing  completion  of  the  software,  an  update  phase 
should  be  planned  to  allow  incorporation  of 
backlogged  change  requests. 


On  the  PTT  project,  the  OAS  was  being  built 
while  TSC,  with  considerable  help  from  SAC, 
was  writing  the  functional  specifications  for  the 
trainer.  This  task  required  an  understanding  of 
how  the  OAS  would  work  in  order  to  define  how  the 
PTT  should  support  the  procedures  identified  as 
training  requirements.  Difficulty  in  obtaining 
and  understanding  existing  OAS  documentation  and 
coping  with  changes  to  the  OAS  resulted  in  a 
significant  schedule  slip.  Working  with  these 
specification  changes  had  one  benefit:  the  soft¬ 
ware  team  recognized  the  need  for  a  flexible 
software  design  to  accoimtodate  the  inevitable 
changes  in  the  future  and  obtained  a  good  under¬ 
standing  of  where  changes  might  occur  in  the  OAS. 

Even  when  the  actual  system  is  stable  through¬ 
out  development  of  the  trainer,  future  changes  to 
the  weapon  system  which  must  be  reflected  in  the 
trainer  are  likely.  Accepting  the  fact  that 
changes  are  unavoidable,  the  trainer  developers 
should  be  encouraged  to  provide  flexibility  for 
future  changes.  Design  tradeoff  decisions  should 
favor  the  straightforward,  easily  modifiable 
approach  over  a  more  efficient  method  requiring  a 
complete  redesign  if  one  of  the  requirements 
changes.  As  an  example,  one  technique  used  by  TSC 
is  to  provide  many  of  the  system  parameters  in 
separate  data  files  that  can  be  easily  changed 
without  affecting  any  of  the  code  which  uses 
this  data. 

Some  of  the  techniques  suggested  for  helping 
to  meet  short  schedules  also  facilitate  incorpo¬ 
rating  requirement  changes:  producing  PDL  ensures 
that  the  software  design  is  documented,  making  it 
easier  to  see  the  impact  of  a  change;  using  a 
high-level  language  and  PDL  as  comments  in  the 
code  makes  the  code  more  understandable,  facilitat¬ 
ing  coding  changes;  and  reviewing  PDL  and  code 
results  in  a  more  flexible  design  and  code  to 
accommodate  future  changes. 


SUMMARY 

We  have  discussed  why  shortened  schedules  and 
changing  requirements  are  often  associated  with  the 
development  of  training  systems.  These  conditions 
increase  the  probability  of  schedule  slips  and 
cost  overruns  or  delivery  of  an  unacceptable 
training  device  if  the  development  plan  does  not 
adequately  provide  means  of  dealing  with  them. 
Several  techniques  used  by  TSC  to  minimize  the 
impact  of  problems  caused  by  schedule  slips  and 
requirement  changes  were  presented.  These 
techniques  are  being  applied  to  the  B-52  OAS  Part 
Task  Trainer,  which  was  briefly  described. 

A  major  technique  for  dealing  with  potential 
schedule  problems  is  top-down  development.  As 
described  with  specific  examples  from  the  PTT, 
top-down  development  entails  the  implementation  of 
successive  versions  of  the  crainer  so  that,  if 
delays  occur,  a  working  version  of  the  partial 
system  is  available  on  the  original  deadline  while 
development  of  the  complete  device  continues.  An 
important  part  of  the  definition  of  versions  for 
top-down  development  is  ranking  the  priority  of 
training  requirements  to  schedule  the  more  critical 
functions  in  the  earlier  versions,  maximizing 
usefulness  of  the  working  versions  while  the 
trainer  is  completed.  Program  design  language  as 
well  as  design  and  code  reviews  are  other 
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T'TLE  BUFFER  KEYBOARD  INPUT  (BKI ) 
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CURSOR  TAB  TASK:  BKI 

POSITION  THE  CURSOR  ON  THE  MENU  IN  ACCORDANCE  WITH  THE 
CURSOR  POSITION  ENTERED  BY  THE  USER. 

A  CIRCULAR  SCHEME  IS  EMPLOYED.  THE  CURSOR  ALWAYS  CIRCLES 
AROUND  IN  THE  SAME  COLUMN  OR  THE  SAME  ROW.  WHEN  AT  THE 
BOTTOM  OF  A  COLUMN,  A  DOWN  TAB  CAUSES  THE  CURSOR  TO 
‘CIRCLE’  TO  THE  TOP  OF  THE  SAME  COLUMN.  WHEN  AT  THE 
TOP  OF  A  COLUMN,  AN  UP  TAB  CAUSES  THE  CURSOR  TO  ‘CIRCLE1  " 

TO  THE  BOTTOM  OF  A  COLUMN.  THE  LEFT 
AND  RIGHT  TABS  HAVE  THE  SAME  EFFECT.  THAT  IS  IF  THERE 
EXISTS  ANOTHER  COLUMN  (POSSIBLE  ONLY  IN  COMMAND  SELECT)' ‘. 

THEN  LEFT  OR  RIGHT  POSITIONS  THE  CURSOR  IN  THE  NEXT 
COLUMN. 

INPUT  PARAMETERS- 

CURRENT  INPUT  STATE  (FOR  BKI) 

COMMAND  ID 
PARAMETER  ID 

CURSOR  KEYSTROKE  (FROM  MOC  KEYBOARD) 

OUTPUT  PARAMETERS: 

COMMAND  ID  (UPDATED) 

PARAMETER  ID  (UPDATED) 

COMMANDS  FOR  THE  MOC  TO  POSITION  CURSOR 
DATA  BASE  USAGE: 

NONE 

INVOKING  METHOD: 

INVOKE  CURSOR  TAB 
INVOKED  BY: 

COMMAND  SELECT  KEYSTROKE  (TO  POSITION  CURSOR) 

PARAMETER  SELECT  KEYSTROKE  (TO  POSITION  CURSOR) 

BLOCKS  INVOKED: 

NONE 

t*******************************************************************************-' 
BLOCK  CURSOR  TAB 
.  IF  DOWN  CURSOR  KEYSTROKE 
.  .  IF  AT  THE  BOTTOM  OF  A  COLUMN 

.  .  .  PUT  CMNDS  IN  MOC  BUFFER  TO  POSITION  CURSOR  AT  TOP  OF  COLUMN 

.  ELSE  :  NOT  AT  THE  BOTTOM  OF  A  MENU 

.  PUT  CMNDS  IN  MOC  BUFFER  TO  POSITION  CURSOR  AT  NEXT  ROW  DOWN 

.  .  ENDIF 

.  ORIF  UP  CURSOR  KEYSTROKE 
.  .  IF  CURSOR  IS  AT  THE  TOP  OF  A  COLUMN 

.  PUT  CMNDS  IN  MOC  BUFFER  TO  POSITION  CURSOR  AT  BOTTOM  OF  COLUMN 
ELSE  :  NOT  AT  THE  TOP  OF  A  MENU 

.  PUT  CMNDS  IN  MOC  BUFFER  TO  POSITION  CURSOR  AT  NEXT  ROW  UP 
.  .  ENDIF 

.  ORIF  KYSTRK  IS  LEFT/RIGHT  AND  NOT  IN  PARAMETER  SELECT  INPUT  STATE 
IF  THERE  ARE  TWO  COLUMNS  &  MENU  COMMANDS  IN  THE  NEXT  COLUMN 
PUT  CMNDS  IN  MOC  BUFFER  TO  POSITION  CURSOR  INTO  NEXT  COLUMN 
ENDIF 

.  ORIF  KEYSTROKE  IS  A  HOME  KEYSTROKE 

PUT  CMNDS  IN  MOC  BUFFER  TO  POSITION  CURSOR  AT  TOP  OF  MENU 
.  ENDIF 

.  IF  CURRENT  INPUT  STATE  IS  COMMAND  SELECT 
.  .  MODIFY  COMMAND  ID 

.  ELSE  :  CURRENT  INPUT  STATE  IS  PARAMETER  SELECT 
.  MODIFY  PARAMETER  ID 

.  ENDIF 

ENDBLOCK  :  CURSOR  TAB 


NAME: 

PURPOSE: 

METHOD: 


Figure  3.  Program  Design  Language  (PDL) 
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techniques  that  help  to  avoid  schedule  slips  and 
facilitate  requirement  changes. 

Programmer  familiarity,  ease  of  use,  and 
availability  of  required  operating  system  functions 
and  software  tools  are  factors  which  should  be 
considered  In  selecting  the  computer  for  the 
training  device  In  order  to  help  meet  shortened 
schedules. 
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ABSTRACT 


This  paper  describes  a  missile  flight  simulator  developed  to  train  DRAGON  gunners.  It  Is 
also  being  adapted  to  a  variety  of  similar  anti-armor  weapons.  The  system  employs  a  terrain 
board  with  enemy  armored  vehicles  moving  in  a  variety  of  attack  scenarios.  When  the  gunner 
fires  the  missile  he  hears  computer  generated  rocket  sounds  and  experiences  the  weight  loss, 
recoil  and  smoke  of  the  missile  launch.  When  the  smoke  clears  he  views  the  missile  as  well 
as  the  target.  The  gunner's  aiming  error  is  measured  using  a  microprocessor  controlled  diode 
matrix  array.  The  matrix  detector  senses  an  IR  emitting  diode  which  is  located  on  the  minia¬ 
ture  target.  The  flight  equations  of  motion  for  the  missile  are  solved  by  a  16  bit  micropro¬ 
cessor  every  0.02  seconds  In  each  axis  using  gunner  aiming  error,  gravity,  drag  and  side 
thruster  accelerations  as  inputs.  A  second  coordinated  16  bit  processor  controls  a  display 
that  plots  both  vertical  and  horizontal  aiming  error  for  analysis  of  the  gunner's  perfor¬ 
mance.  Experienced  DRAGON  gunners  have  tested  the  system  and  attested  to  the  realism  and 
training  potential. 


INTRODUCTION 

Training  in  the  firing  of  modern  anti-armor 
weapons  is  expensive.  Each  live  round  costs 
thousands  of  dollars. 

This  paper  describes  a  system  that  uses 
advanced  electro-optics  and  microprocessor  tech¬ 
nology  to  enable  training  of  DRAGON  gunners  at  a 
reasonable  cost. 

The  DRAGON  is  a  command-to-line-of-sight 
guided  missile  system.  Fired  from  a  recoilless 
launcher,  the  missile  is  tracked  optically  and 
guided  automatically  to  the  target  by  electrical 
impulses  transmitted  via  a  wire  link.  Firing 
the  missile  is  accomplished  by  depressing  the 
safety  and  squeezing  the  trigger.  No  other 
action  is  required  of  the  gunner  except  to  keep 
the  sight  cross  hairs  on  the  target.  However, 
to  score  a  hit  the  trainee  must  overcome  many 
perturbations  that  can  spoil  his  track. 

When  the  trainee  fires  the  training  device 
he  hears  the  gyro  wind-up  noise  and  then  the 
initial  explosion  of  the  rocket  motor.  He 
experiences  a  weight  loss  due  to  the  rocket 
exiting  the  tube  as  well  as  a  recoil  force. 
Momentarily  he  is  blinded  in  the  sight  by  simu¬ 
lated  smoke.  The  trainee  must  overcome  such 
launch  transients.  He  must  then  track  the  tar¬ 
get  smoothly  and  ignore  the  simulated  missile 
which  he  can  see  in  his  sight.  Thruster  rocket 
firing  sounds  are  included  as  well  as  the  final 
hit  or  ground  impact  explosions.  A  visual  indi¬ 
cation  of  hit  is  also  inserted  into  the  gunners 
sight. 


During  missile  flight  the  instructor  can 
monitor  two  displays.  These  displays  show: 

1.  The  gunner's  sight  picture  and  the 
DRAGON'S  location 

2.  A  plot  of  gunner  aiming  error  versus 

time  and  the  gunner  error  tracking  limit  enve¬ 
lopes.  Thruster  firings  are  annotated  on  the 

display. 

This  system  uses  a  16  bit  microprocessor  to 
solve  the  flight  equations  every  0.02  seconds  in 
each  axis  using  the  gunner's  aiming  error, 
gravity,  drag  and  thruster  rocket  acceleration 
as  inputs.  The  solution  also  incorporates  the 
dynamic  performance  of  the  tracker  (See  Artist's 
concept) . 

Key  features  of  the  system  are  summarized 

below. 

o  Smoke  Obscuration 

o  Recoil 

o  Weight  Loss 

o  Missile  superimposed  on  gunner's  view 

of  scenario 

o  Sounds  -  thruster  firings,  launch,  hit 

and  miss  explosions 

o  Gunner  aiming  errors  versus  time  dis¬ 

played  in  real  time 
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o  Can  operate  for  night  scenes  to  simu¬ 
late  a  thermal  sight 

o  Missile  position  versus  time  which  can 
be  recalled  along  with  gunner  aiming 
errors  In  azimuth  and  elevation  for 
analysis 

o  Cost  of  expensive  tank  target  and  mis¬ 
sile  not  required  for  training 

o  Target  hit  or  miss  distance  determined 
by  solving  DRAGON  flight  equations  In 
real  time 

o  Number  of  thruster  rockets  Ideal  versus 

actual  displayed  for  each  scenario 

o  Portable 

o  Record  and  play  back  capability 

o  Can  operate  with  and  without  an  in¬ 

structor 

o  Operator's  pull  down  force  on  DRAGON 

launcher  and  eye  cup  pressure  is 
measured 

o  Variety  of  target  speeds  and  motions 

simulated 

o  Trainer  flies  like  real  missile  because 

of  computation  of  flight  parameters 

The  system  has  been  tested  by  both  U.S.  Army 
and  U.S.  Marine  Corps  DRAGON  gunners.  Further 
development  of  the  trainer  is  now  being  accele¬ 
rated. 

SYSTEM  APPROACH 

The  system  Is  shown  in  Figure  l. 


dlBplay  plota  CAE  versus  time,  In  real  time. 
The  DRAGON  Flight  Simulator  Processor  produces 
launch  and  target  explosions ,  thruster  rocket 
firings  and  gyro  noises.  The  thruster  rocket 
firings  are  delayed  to  allow  for  the  speed  of 
sound  versus  the  vlaual  phenomena  of  the  rocket 
firing  which  la  optically  Inserted  in  the  DRAGON 
gunner's  sight.  Rocket  thruster  nolaes  are 
attenuated  aa  a  function  of  distance. 

A  CCTV  la  located  on  the  DRAGON  tube  and 
boresighted  to  the  gunner's  6X  sight.  This  TV 
provides  the  Instructor  the  same  view  seen 
through  the  gunner's  sight.  The  Gunner's  Sight 
Picture  Display  is  located  on  the  instructor's 
console.  The  DRAGON  rocket  as  seen  by  the 
trainee  is  also  mixed  into  the  gunner's  sight 
picture  visual  display. 

SUBSYSTEM  DESIGN  APPROACH 
Electro  Optics  Subsystem 

Gunner  aiming  errors  are  determined  using  a 
100  x  100  matrix  camera.  Functionally  the  cam¬ 
era  is  similar  to  a  Vldicon  camera  except  that 
the  aenBor  has  been  replaced  with  a  solid  atatt 
photodiode  array  matrix  having  10,000  pixels. 
The  choice  of  lens  determines  the  field  viewed 
by  the  camera.  Using  a  125mm  focal  length  lens 
and  a  model  distance  of  22  feet,  the  available 
field  of  view  is  1.05  ft  or  48  mllllradlans. 
This  F0V  will  accommodate  the  maximum  excursions 
allowed  for  DRAGON  i.e.,  32  mr  horizontal  and 
22  mr  vertical. 

For  a  1.05  ft  FOV  one  pixel  represents  0.126 
inches  on  a  terrain  board. 

Since  the  array  is  square  the  lengths  in  the 
X  and  Y  axes  are  identical.  The  magnification 
of  the  camera  is  the  ratio  of  the  FOV  to  the 
length  of  the  arrays 


Targets  in  this  system  are  miniature 
models.  Models  were  chosen  because  they  have 
better  resolution  than  either  computer  generated 
imagery  or  a  movie  display.  In  DRAGON  a  6X 
scope  Is  utilized.  In  other  weapons  even  higher 
power  sighting  scopes  are  utilized,  thus  demand¬ 
ing  a  high  resolution  visual  scenario. 

Models  are  moved  on  a  terrain  board  using  a 
stepper  motor  under  the  control  of  a  single  chip 
microprocessor.  Located  at  the  center  of  aim  of 
the  target  is  an  infrared  emitting  diode 
(IRED).  Engagement  scenarios  are  stored  in  the 
Personnel  Interface  Processor  (PIP)  and  one  is 
selectable  from  the  instructor's  console  by  an 
input  terminal.  The  stored  scenario  programs 
contain  the  tank  target's  velocity,  direction 
and  range.  Located  in  the  DRAGON  launch  tube  is 
a  photo  diode  array  camera.  This  100  x  100 
matrix  camera  is  boresighted  to  the  gunner's 
sight  and  used  to  determine  the  gunner's  aiming 
error  (GAE)  which  is  input  to  the  DRAGON  Flight 
Simulator  Processor  (DFS).  This  processor 
solves  the  DRAGON  flight  equations  and  provides 
DRAGON  status  to  the  Personnel  Interface  Pro¬ 
cessor  (PIP).  The  PIP  controls  the  graphics 
units  which  inserts  the  missile,  smoke,  hit, 
etc.,  into  the  gunner's  sight.  Thin  processor 
also  controls  the  Gunner  Aiming  Error  (GAE) 
display  on  the  Instructor's  Console.  This 
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Magnification 


FOV 

Array  Length 


where  the  array  length  is  *  0.24  in. 
total  width/height)  In  both  X  and  Y. 

.05  x  12 


(0.60  cm 


M 


52.5 


0.24 


The  static  resolution  is  the  array  element 
spacing  imaged  into  the  object  plane. 

Resolution  ■  Magnification  x  element  spacing 

Resolution  •  52.5  x  0.0024  in  ■  0,126  in. 


This  is  equivalent  to  +  7.5  inch  resolution 
on  a  real  world  tank  at  a  scaled  range  of  2640 
feet. 


This  means  the  smallest  detectable  change  in 
a  stationary  object  we  can  detect  is  0.126 
inches  using  a  i25mm  focal  length  lens.  If  a 
longer  focal  length  lens  is  used  the  FOV  is  de¬ 
creased  and  the  resolution  is  improved. 

Accuracy  also  depends  on:  image  sharpness, 
contrast,  vibration  or  movement  of  the  object, 
light  level  and  threshold  setting  of  the  cam¬ 
era. 
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The  earner*  used  la  blemish  free. 

An  IRED  la  located  on  th#  target  and  th* 
center  of  the  IRED's  energy  calculated  to  deter¬ 
mine  hit  location. 

Because  the  IRED  produces  uniform  Illumina¬ 
tion,  the  threshold  setting  on  the  camera  can  be 
adjusted  to  a  fixed  level,  thus  eliminating 
background  Interference. 

Data  from  the  photodiode  array  arc  elec¬ 
tronically  scanned  to  produce  a  aampled-and-held 
video  output  signal.  The  amplitude  of  each 
pixel  Is  proportional  to  the  incident  light  In¬ 
tensity  integrated  over  th*  Interval  of  on* 
frame  period.  The  camera  essentially  detects 
light  to  dark  transitions  of  the  digital  area. 
The  scene  present  on  the  camera  la  a  light  cir¬ 
cle  on  a  dark  background.  Tranaltlon  data  from 
the  camera,  stored  as  a  digital  llne-by-lin* 
picture  of  the  array,  Is  handled  by  an  Interface 
unit.  The  DRAGON  Flight  Simulator  Processor 
determines  the  GAE  from  the  transition  data. 

Microprocessor  Subsystem 

The  microprocessor  subsystem  Includes  six 
units  with  five  being  housed  In  the  system 
chassis.  The  principal  function  of  each  of  the 
separate  units  Is: 

1.  Personnel  Interface  Processing  (PIP) 

2.  DRAGON  Flight  Simulation  (DFS) 

3.  Sound  Generation  (SG) 

4.  Target  Control  (TC) 

5.  TV  Display  (TVD) 

6.  Photodiode  Array  Processing  (PAP) 

System  I/O  is  processed  by  the  PIP,  which  is 
covered  in  the  Computer  Graphics  and  Video  Sub¬ 
system  section. 

Target  control  is  detailed  in  the  Miniature 
Target  Board  section. 

Descriptions  of  the  DRAGON  Flight  Simulator 
and  the  Photodiode  Array  Processor  follow  in  the 
next  two  sections. 

DRAGON  Flight  Simulator 

The  McDonnnell  Douglas  Astronautics  Company, 
Titusville  Division,  under  Contract 

N61339-80-M-3518  provided  a  set  of  simplified 
equations  and  a  computer  program  that  approxi¬ 
mate  the  DRAGON  missile  flight  as  directed  by 
the  gunner. 

Six-degree-of-freedom  equations  are  required 
to  express  the  complete  missile  dynamics.  Solu¬ 
tions  of  such  equations  were  examined  and  sim¬ 
plified  as  much  as  possible  by  McDonnell  while 
still  maintaining  a  statistically  accurate  rep¬ 
resentation  of  weapon  performance*  Some  of  the 
simplifying  assumptions  were: 


1.  Missile  dynamics  should  b*  represented 
by  a  point  mass  solution, 

2.  Small  angle  approximations  to  be  used, 

3.  The  effect  of  tracker  sampling  on  mis¬ 
sile  trajectory  while  In  the  linear  field  of 
view  to  be  neglected. 

The  six-degree-of-freedom  equations  thus  modi¬ 
fied  were  exercised  and  compared  to  results 
obtained  from  the  complete  equations  of  motion. 
Modification  to  the  thrust  level  and  guidance 
parameters  were  mad*  to  tailor  th*  trajectory  to 
the  more  exact  results.  Suiflclent  comparative 
analysis  was  conducted  to  assure  that  the  sim¬ 
plified  equations  gave  acceptable  results  over  a 
range  of  crosstng  and  stationary  target  condi¬ 
tions  and  with  a  variety  of  gunner  aiming 
errors. 

Figure  2  is  the  DRAuON  simulation  block  dia¬ 
gram.  The  variables  correspond  with  those  of 
Figure  3  which  defines  the  important  horlaontal 
angles.  These,  and  a  similar  set  of  vertical 
cnglea,  were  used  in  the  McDonnell  BASIC  program 
which  iterates  the  differential  equations  of  mo¬ 
tion  using  a  "Delta  Time"  of  20  milliseconds* 
Thus  a  10  second  missile  flight  requires  the 
generation  of  500  solutions  of  th*  equations  of 
motion* 

The  BASIC  program  was  rewritten  for  an  Intel 
Microprocessor  Development  (MDS)  System.  The 
resulting  program,  while  able  to  reproduce  the 
McDonnell  results,  required  several  minutes  to 
complete  the  500  solutions  for  a  simulated  10 
second  missile  flight.  It  was,  therefore,  un¬ 
suitable  for  real  time  training. 

An  investigation  of  other  floating-point- 
math  techniques  usable  with  Intel  SBC-86/12, 
8086,  computers  showed  that  real-time  solutions 
of  the  missile  flight  could  not  be  accomplished 
without  using  an  8087  coprocessor.  The  non¬ 
availability  of  the  8087  made  it  necessary  to 
abandon  the  convenience  of  FP-math  and  recast 
the  equations  using  Integer  arithmetic.  This 
required  close  attention  to  the  choice  of  suit¬ 
able  units  for  the  variables  because  of  the 
limited  range  of  integer  numbers:  (-32,767, 

+32,767).  Down-range  distances,  for  example, 
are  expressed  in  2-inch  units;  1000  meters 
(39,370  inches)  being  considered  to  be  19,685 
"Down-range"  units.  Cross-Orange  units  are  0.05 
inches  for  distances  and  0*1  milliradlans  for 
angles.  Unit  selection  is  a  compromise  between 
the  conflicting  requirements  of  the  desire  to 
display  variables  over  a  wide  range  and  the  need 
to  reduce  the  quantisation  distortion  while  not 
exceeding  the  allowable  integer  range.  Many 
comparisons  between  the  integer  and  BASIC  pro¬ 
gram  results  have  verified  that  good  approxima¬ 
tions  to  the  DRAGON  Flight  characteristics  are 
provided  using  integer  arithmetic.  Comments  by 
experienced  DRAGON  gunners  also  support  the  va¬ 
lidity  of  the  approximations. 

The  DRAGON  Flight  Simulation  Program  in¬ 
cludes  five  modules: 

1.  Main-DRAGON-Module:  A  “Driver"  module 
which  calls  other  modules. 
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2.  DRACON-ut lllty :  Includes  a  number  of 
start-up  and  other  general  procedures < 

3.  DRAGON  Flight  Module:  Includes  the 
Integer  math  missile  dynamics,  provides  missile 
location  Information  to  the  PIP,  stores  location 
data  for  possible  reprise,  and  does  the  initial¬ 
isation  of  flight  variables, 

4.  DRAGON  IR:  Analyses  the  IR-spot  data 
array  provided  by  the  DRAGON  XF  module, 

5.  DRAGON  XF:  Transfers  line-by-line  data 
provided  by  the  photo-detector  line  array  pro¬ 
cessor  into  a  complete  picture  array. 

The  first  three  modules  are  written  in  PLM 
86;  an  Intel  high  level  programming  language. 
The  last  two  are  in  8086  assembly  language. 
Total  program  code  require  slightly  under  4K  of 
ROM  memory.  Variable  memory  requires  about  IK 
of  RAM. 

As  noted  previously,  the  program  ROM  is  lo¬ 
cated  on  an  Intel  SBC  86/12  board.  This  board, 
along  with  four  others  are  housed  in  an  Intel 
SBC  86/12  system  chassis  which  provides  eight 
card  slots,  power  supply  and  ventilation.  Cards 
within  the  chassis  can  communicate  via  the  mul¬ 
tibus  motherboard.  An  SBC  86/12  provides  dual¬ 
port  RAM  which  can  be  accessed  by  both  the 
on-and-of f-board  processors.  Missile  position 
data  resulting  for  the  solution  of  the  missile 
equations  of  motion  are  transferred  to  the  PIP 
via  the  multibus  for  further  processing  and  out¬ 
put.'  Data  status  bits  are  also  read  and  written 
across  the  multibus  as  required. 

Target  motion  is  provided  as  described  in 
the  section  on  Miniature  Modelboard.  It  is  pro¬ 
grammed  via  a  stepper  motor  controller  into 
which  the  desired  target  maneuver  is  input  from 
a  suitable  menu  item  located  in  program  memory 
of  the  PIP.  Identification  of  the  selected 
maneuver  is  posted  within  dual-port  memory  and 
therefore  may  be  read  by  the  DFS  in  order  to 
make  possible  appropriate  target  position  calcu¬ 
lations  aa  required  by  the  missile  equations  of 
motion. 

The  DFS  also  provides  control  signals  to  the 
sound  generator  for  side-thruster  pops,  ground 
explosion  and  target  hits.  It  also  provides 
signals  for  weight  loss  in  response  to  trigger 
pull. 

Photodiode  Array  Processor 

Line  scan  data  from  the  100  x  100  photodiode 
array  are  initially  stored  in  a  set  of  ping-pong 
memories  on  a  Retlcon  RSB  6020  board  housed 
within  the  system  chassis  and  attached  to  the 
multibus.  Data  are  alternately  read  into  ping 
or  pong  memory  under  control  of  a  clock  located 
within  the  Reticon  RS  520.  Data  within  the 
memory  units  gives  the  location  of  light  level 
transitions  and  indicates  whether  it  is  a 
light-dark  or  dark-light  transition.  The  stored 
data  also  indicate  when  the  last  scan  line  is 
read. 

After  initialization,  a  last-line  flag  is 
output  across  the  multibus  to  the  DFS  which 
causes  the  DRAGON  XF  program  to  begin  the 


transfer  of  data  from  each  line  of  the  next  100 
x  100  photodiode  erray  frame.  The  deta  read-out 
is  then  halted  by  the  next  occurrence  of  the 
leet-llne  flag.  The  100  x  100  frame  date  are 
Ignored  during  the  next  frame  dete  analysis. 
New  frame  data  are  thus  provided  every  other 
frame. 

The  frame  rate  of  the  Retlcon  camera  is  100 
frames  per  Becond  so  new  IR-spot  position  data 
are  provided  50  times  a  second  or  with  e  20 
millisecond  period.  Occurrence  of  the  last-line 
flag  acts  as  the  master  system  clock  with  all 
data  processing  starting  with  its  assertion. 

Computer  Graphics  and  Video  Subsystem 

The  DRAGON  computer  graphic  visual  presenta¬ 
tion  io  prepared  by  the  Personnel  Interface  Pro¬ 
cessor.  In  addition  to  this  processor  a  com¬ 
puter  graphics  board,  a  phase-locked-loop  sync 
board,  and  an  EIA  composite  sync  generator  is 
used.  Figure  4  shows  the  complete  graphics  and 
the  video  subsystem. 

Computer  generated  graphics  provide  two 
major  functions: 

1.  Real-time  video  graphics  are  generated 
for  the  gunner  sight.  These  graphics  Include  a 
simulated  missile  which  Includes  thruster 
firings,  smoke  obscuration  during  initial  launch 
and  a  final  explosion* 

2.  Real-time  graphics  are  generated  for 
the  instructor  which  indicate  both  vertical  and 
horizontal  gunner  aiming  errors.  Also,  for  fol¬ 
low  up  analysis,  graphics  may  be  presented  for 
gunner  aiming  error  versus  time  and  missile  po¬ 
sition  versus  time. 

Gunner's  sight  computer  graphics  are  gene¬ 
rated  on  a  256  x  256  x  4  graphics  board.  Six¬ 
teen  levels  of  gray  scale  provide  for  a  full 
range  of  visual  intensity  which  allows  for  smoke 
generation  which  varies  from  fully  transparent 
to  completely  opaque.  The  computer  generated 
graphics  are  passed  directly  to  the  gunner's 
sight  through  a  one  and  a  quarter  inch  closed 
circuit  television  (CCTV)  monitor.  The  optical 
arrangement  is  shown  in  Figure  5.  The  televi¬ 
sion  screen  appears  at  infinity  along  with  the 
viewed  scene  through  the  6x  scope.  The  CCTV  is 
mounted  inside  the  DRAGON  IR  tracker  housing  and 
electronics  for  the  CCTV  are  located  where  the 
IR  tracker  electronics  were  located  at  the  bot¬ 
tom  of  the  tracking  unit. 

The  instructor  console  graphics  subsystem  is 
composed  of  two  units,  a  television  representa¬ 
tion  of  the  gunner's  sight  picture  and  a  graphi¬ 
cal  plot  of  gunner  aiming  error  versu'.  time 
and/or  gunner  aiming  error  versus  missile  posi¬ 
tion. 

The  television  representation  of  the  gun¬ 
ner's  sight  is  accomplished  by  mixing  the  gun¬ 
ner's  sight  TV  camera,  which  is  boresighted  to 
the  6x  gunner  sight,  with  the  video  graphics 
presented  to  the  gunner's  sight.  The  composite 
picture  presents  to  the  instructor  an  image  of 
the  gunner's  sight  which  Includes  the  target, 
missile,  smoke,  crosshairs  and  final  explosion. 
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The  graphical  plot  of  :  he  gunner  aiming  er¬ 
ror  (GAE)  versus  time  for  both  horizontal  and 
vertical  error  are  presented  In  real-time  during 
the  missile  flight.  The  graphs  Indicate  the 
actual  gunner  aiming  error  during  the  flight  as 
well  as  the  limits  for  a  95%  probability  of  hit 
performance.  The  guidance  rocket  thruster  fir¬ 
ings  are  shown  when  they  are  fired  as  well  as  a 
final  actual  count  of  the  thrusters  fired  versus 
the  Ideal  number  of  thrusters  that  would  have 
been  fired  for  a  given  target  distance  with  per¬ 
fect  aim.  At  the  end  of  a  flight  displayed  re¬ 
sults  show  the  miss  distance,  in  feet,  where  the 
missile  passed  the  target.  If  the  missile 
strikes  the  ground  before  passing  the  target,  a 
message  is  displayed  stating  "ground  Impact”  as 
well  as  the  remaining  distance  to  the  target 
when  grounded.  If  a  hit  Is  scored  i  hit  message 
is  displayed  to  mark  the  event. 


Psuedo-random  noise  may  be  mixed  to  any  or 
all  channels  from  a  noise  generator  with  basic 
frequencies  of  A  KHz  to  125  KHs.  Two  modes  of 
output  control  are  available  for  each  channel. 
The  fixed  level  amplitude  mode  selects  an  ampli¬ 
tude  specified  in  the  array  by  the  microcompu¬ 
ter.  For  use  in  thlB  system  the  variable  ampli¬ 
tude  mode  1 8  selected,  forcing  an  envelope 
generator  to  control  the  shape  and  cycle  of  all 
outputs.  Controlling  the  envelope  generator  is 
a  lb  bit  tone  period  within  the  array  allowing 
'  r  frequency  ranges  of  12  Hz  to  7812.5  Hx  and  a 
,lve  bit  Bhape/cycle  control  register.  Three 
D/A  converters  supply  0  to  1  volt  signals  to  the 
output  channels. 

To  accurately  represent  the  flight  of  a 
DRAGON  ml 88 lie  as  it  moves  down-range  two  sound 
phenomena  must  be  simulated: 


After  a  missile  flight  a  reprise  of  the 
flight  may  be  called.  A  horizontal  reprise  re¬ 
plays  the  horizontal  GAE  and  the  horizontal  mis¬ 
sile  position  versus  time.  Likewise  the  verti¬ 
cal  reprise  replays  the  vertical  GAE  and  the 
vertical  missile  position  versus  time.  The  re¬ 
prises  indicate  all  the  hlt/miss  summaries  of 
the  first  real-time  plot. 

Any  of  the  computer  graph  tc  plots  may  be 
made  into  a  hard-copy  printout  The  hard-copy 
may  include  the  gunner's  name  or  other  pertinent 
data  as  desired  by  the  Instructor. 

Computer  Generated  Sound  System 

Simulation  of  sounds  produced  during  an 
actual  DRAGON  missile  firing  is  accomplished  by 
Interfacing  an  Intel  8748  microcomputer  to  a 
General  Instruments  AY-3-8910  Programmable  Sound 
Generator  (PSD).  Data  necessary  for  the  PSG  to 
reproduce  sounds  is  acquired  from  the  permanent 
memory  of  the  microcomputer.  During  missile 
flight  time  the  DFS  processor  simply  selects  the 
sound  to  be  made  and  communicates  its  choice  to 
the  microcomputer.  This  approach  allows  the 
processor  to  handle  sound-making  decisions  with 
minimum  time  taken  from  its  primary  functions. 

The  choice  of  sounds  available  to  the  DFS 
processor  are: 

1.  Gyro  start-up 

2.  Missile  launch  explosions 

3.  Rocket  thruster  motor  firing 

4.  Target  missed  explosions 

5.  Target  hit  explosions. 

The  General  Instruments  Programmable  Sound  Gene¬ 
rator  (PSG)  is  a  40  pin,  eight  bit  device  with 
microprocessor  compatibility.  The  device  fea¬ 
tures  three  independent  analog  channels  each 
with  access  to  its  own  tone  generator.  A  16 
control  register  array  communicates  to  the 
microcomputer  through  an  eight  bit  bi-direction¬ 
al  port.  Four  lines  are  allotted  for  bus  con¬ 
trol  logic  (read  and  write).  Each  tone  gene¬ 
rator  looks  to  two  registers  within  the  array 
for  a  12  bit  tone  period.  A  range  of  frequen¬ 
cies  covering  the  full  eight  octaves  of  the 
equal  tempered  chromatic  scale  is  available. 


1.  Time  delay  due  to  the  difference  in  the 
speeds  of  light  and  sound,  and 

2.  Logarithmic  sound  amplitude  decay  due 
to  distance  sound  must  travel  through  air. 

Software  developed  for  the  microcomputer  closely 
approximates  these  conditions  within  a  1000 
meter  range. 

As  shown  in  Figure  6,  the  outputs  of  the 
PSGs  are  input  to  circuits  which  function  to 
control  the  amplitude  of  the  sound.  These  cir¬ 
cuits  consist  of  operational  amplifiers  with 
closed  loop  gains  under  direct  control  of  the 
microcomputer.  The  DRAGON  Flight  Simulator  pro¬ 
cessor  initiates  a  timer  within  the  microcompu¬ 
ter  upon  request  of  a  launch  explosion.  There¬ 
after,  each  request  for  a  sound  by  the  processor 
causes  the  microcomputer  to  inspect  the  timer. 
Assuming  the  missile  travels  at  an  average  speed 
of  280  feet  per  second  the  microcomputer  is  able 
to  approximate  the  distance  covered  and  set  the 
appropriate  gain.  For  rocket  thrust-,  r  firings, 
the  microcomputer  selects  one  of  thirteen  levels 
of  amplitude,  decreasing  logarithmically  from  a 
gain  of  ten  to  one  over  a  time  span  of  11 
seconds  corresponding  to  a  distance  of  1000 
meters. 

Time  delay  associated  with  distance  covered 
by  the  missile  is  accomplished  upon  inspection 
of  the  timer  for  each  requested  sound  after 
launch.  Before  signals  are  passed  to  the  PSG  to 
create  a  sound,  software  completes  a  sequence  of 
three  delays.  The  first  delay  represents  the 
real-time  between  requests  from  the  DFS  pro¬ 
cessor.  This  timeout  occurs  only  when  two  or 
more  requests  are  made  before  the  first  request 
is  serviced  by  passing  signals  to  the  PSG.  The 
real-time  between  any  two  requests  represents 
distance  traveled  by  the  missile  and  is  decoded 
into  the  second  time  delay  as  determined  by  the 
time  required  for  sound  waves  to  travel  this 
distance.  The  incremental  time  delays  are  ac¬ 
cumulated  in  the  microcomputers  data  memory. 
The  third  time  delay  before  a  sound  is  made  is 
the  cumulative  total  of  all  the  second  time 
delays  that  have  already  been  decoded.  The  com¬ 
plete  algorithm  produces  a  series  of  logarithmi¬ 
cally  decaying,  time  delayed,  sound  waves  that 
approximate  the  actual  conditions  within  a  1000 
meter  range. 
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Miniature  Target  Board 

Because  moat  anti-armor  devices  use  high 
power  telescopes  to  view  the  targets,  a  minia¬ 
ture  model  was  chosen.  The  target  model  has  an 
IRED  iucatec  at  the  center  of  the  target  mass. 
The  mode!  is  moved  using  a  stepper  motor.  The 
stepper  motor  controller  Is  a  stand-alone  in¬ 
telligent  controller  that  Is  Independent  of  the 
host  computer,  the  Personnel  Interface  Pro¬ 
cessor,  except  for  loading  the  scenario.  The 
stepper  motor  controller  uses  a  high  level  lan¬ 
guage  for  control  of  the  stepper  motors  direc¬ 
tion,  position,  speed  and  acceleration.  Scaled 
to  the  real  world  the  tank  location  is  known  to 
0.9  Inches. 

Weight  Loss  and  Recoil  Mechanism 

Launch  effects  of  the  DRAGON  simulator  are  a 
very  Important  facet  of  the  training  mission. 
Two  of  the  launch  transients  which  must  be  over¬ 
come  by  the  DRAGON  gunner  are  the  weight  loss 
due  to  the  missile  leaving  the  launch  tube  and 
the  recoil  of  the  launcher  due  to  slight  uncom¬ 
pensated  differences  in  the  pressures  at 
launch.  Weapon  launch  effects  of  weight  loss 
and  recoil  ar..-  simulated  via  mechanical  attach¬ 
ments  to  the  DRAGON  bipod. 

The  recoil  mechanism  Is  a  sliding  platten 
upon  which  the  DRAGON  bipod  and  gunner's  feet 
are  supported.  The  platten  is  covered  with  a 
rubber  and  steel  hybrid  material  that  allows  the 
gunner  to  firmly  plant  the  bipod  legs  in  posi¬ 
tion  and  stabilize  the  launcher  using  his  boots 
to  press  against  the  bipod  supports.  At  launch 
the  platten  Is  given  an  Impulse  from  a  pneumatic 
solenoid  thus  imparting  a  sensation  of  recoil  to 
the  launcher. 

The  weight  loss  simulation  Is  accomplished 
by  a  weight  mass  that  Is  attached  to  the  bipod 
via  a  p.vot  and  pneumatic  cylinder.  When  the 
DRACON  simulator  Is  armed  for  launch,  the  pneu¬ 
matic  cylinder  Is  energized  which  in  turn  raises 
the  weight  and  places  an  additional  equivalent 
weight  of  the  DRAGON  missile  on  the  shoulder  of 
the  DRAGON  gunner  through  mechanical  leverage. 
When  the  simulated  missile  is  launched,  the 
pneumatic  cylinder  Is  relaxed,  thus  releasing 
the  weight  and  effectively  removing  the  equiva¬ 
lent  missile  weight  from  the  gunner's  shoulder. 

CONCLUSION 

This  simulator  has  undergone  preliminary 
evaluation  by  a  United  States  Marine  Corps  Fleet 
Project  Team  of  experienced  DRAGON  gunners.  All 
gunners  were  favorably  impressed  with  its  real¬ 
ism  and  teaching  attributes.  Testing  of  the 
device  Is  planned  for  the  fall  of  1981  by  both 
the  United  States  Marine  Corps  and  the  United 
States  Army.  Results  of  these  tests  will  be 
reported  at  the  conference. 
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FIGURE  3.  HORIZONTAL  PLANE  GEOMETRY 
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FIGURE  4.  COMPUTER  GRAPHICS  AND  VIDEO  SUBSYSTEM 
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FIGURE  6.  SOUND  GENERATOR  HARDWARE  LAYOUT 


PHANTOM  RANGE  -  AN  EW  TRAINING  SY8TEM 


Morton  T.  Eldridge 
Teledyne  Brown  Engineering 
Huntsville,  Alabama 


ABSTRACT 

Confronting  NATO  tactical  air  is  a  spectrum  of  Warsaw  Pact  defoiiS££  including  SAMs,  AAA,  and 
airborne  interceptors  plus  jamming  of  communications ,  fire  control  radars,  and  navigation 
equipment.  Aircrews  faced  with  this  array,  trying  to  perform  their  primary  mission,  must  be 
trained  to  cope  with  the  total  anticipated  task-loading  and  at  the  same  time  become  neither 
casualties  nor  disoriented  such  that  they  fail  to  achieve  their  mission  objective.  Current 
training  on  large  EW  ranges  is  considered  inadequate  due  to  the  limited  accessibility  and  the 
infrequency  with  which  aircrews  can  experience  such  training.  The  Phantom  Range,  an  onboard, 
computer-generated  threat  simulator,  can  be  programmed  to  provide  threats  at  given  geographic 
locations,  independent  of  ground  emitters,  with  appropriate  envelopes  modified  by  actual  existing 
terrain.  It  allows  the  aircrew  to  defeat  the  threat  by  exercising  proper  procedures,  or  be 
"killed"  if  their  actions  are  inappropriate.  The  whole  scenario  is  recorded  for  ground 
debriefing. 


INTRODUCTION 

Phantom  Range  is  an  onboard,  computer- 
generated  threat  simulation  system  that  enables  air¬ 
crews  to  interact  with  defensive  scenarios  during 
actual  combat  training  missions  and  perfect  their 
techniques  in  exercising  appropriate  defeat  pro¬ 
cedures  independent  of  ground  emitters  (Pigure  1). 
The  system  offers  the  capability  to  incorporate 
scenarios  of  varying  complexity,  in  terms  of  varying 
locations,  numbers,  and  kinds  of  threats.  It  pro¬ 
vides  real-time  feedback  of  success  or  failure  in 
accomplishing  the  proper  procedures  in  a  timely 
manner.  Finally,  it  records  encounters  so  that  the 
aircrew  can  debrief,  observing  the  actual  displays 
that  were  seen  during  each  engagement . 


THREAT 

Readiness  of  tactical  air  forces  combines  the 
availability  of  weapons  systems,  ordnance,  and 
logistics  support;  the  inherent  capability  of  the 
weapons  system  itself;  and  most  important,  the 


ability  of  the  aircrews  to  cope  with  all  of  the  com¬ 
posite  stresses  of  a  combat  situation  and  still 
function  effectively  to  perform  their  primary  offen¬ 
sive  missions  (Figure  2).  This  last,  considering 
the  anticipated  threat,  ia  a  momentous  task. 

The  threat  consists  of  a  very  dense  and  effec¬ 
tive  array  of  surface-to-air  weapons  (SAMs  and  AAA), 
airborne  interceptors,  and  electronic  counter¬ 
measures  (Figure  3).  Communications  jamming  not 
only  impairs  coamand  and  control,  it  also  serves  as 
an  annoyance  and  a  distraction  to  aircrews.  Jamming 
of  forward-looking  radars  inhibits  the  ability  to 
use  this  system  for  terrain  avoidance  or  following. 
And  of  course  jamming  of  navigation  systems,  or  even 
worse,  sending  of  false  signals,  makes  radio  naviga¬ 
tion  systems  unreliable  or  erroneous.  Together, 
these  add  significantly  to  the  task-loading  of 
crews  —  many  in  single-seat  fighters  —  crews  who 
are  already  heavily  stressed  just  to  navigate,  main¬ 
tain  flight  integrity,  and  accurately  deliver  their 
ordnance . 


FIGURE  2.  THE  COMPOSITE  STRESSES  OF  A  COMBAT 
FIGURE  1.  PHANTOM  RANGE  SITUATION 
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SURFACE  TO  AIR  MISSILES  -  DENSE,  IN-DEPTH,  NETTED 
AAA  ■  RADAR  CONTROLLED,  ACCURATE 

AIR  INTERCEPTORS  -  INTEGRATED  DEFENSE  (LESS  THREAT  TO  LOW  RVERSI 

COMMUNICATIONS  JAMMING  •  INTERFERES  WITH  FLIGHT  INTEGRITY  AND  CONTROL, 
DISTRACTING 

AIIFLRINAVICATION  JAMMING  ■  INCREASED  PILOT  WORKLOAD  AND  VULNERAS1LITY 

RESULT:  DEGRADED  NATO  AIR  EFFECTIVENESS 
LOST  AIRCRAFT/AIRCREWS 
DISORIENTATION  AND  MISSION  ABORT 


FIGURE  3.  THREAT 


READINESS  OBJECTIVE 

To  achieve  the  requisite  reedineti,  aircrew* 
oust  learn  to  accoomodate  all  of  the  atressea  and 
distractions  and  still  perform  effectively.  They 
must  be  able  to  do  the  following: 

e  Navigate  by  dead  reckoning  only 

e  Reorient  themselves  and  still  reach  their 
targets  after  performing  defensive  maneuvers  that 
have  driven  them  off  their  planned  route 

e  Maintain  flight  integrity.  Flight  leaders 
must  be  able  to  direct  the  flight  even  though  radio 
communications  is  unfeasible. 

e  Learn  to  respond  to  threats  in  a  timelyi 
correct  manner  so  as  to  minimise  lost  potential 

e  Concentrate  on  employing  their  weapon 
systems  while  responding  to  defensive  threats  and 
ignoring  distractions. 

In  summary,  they  must  become  completely  accli¬ 
mated  to  the  conditions  of  the  defensive  scenario 
such  that  they  can  continue  to  function  effectively 
in  carrying  out  their  primary  mission. 


TRAINING  REQUIREMENTS 

To  achieve  the  above,  the  training  program  must 
provide  realistic  scenarios,  crew  interaction,  and 
continuation  training  (Figure  4) . 


Realistic  Scenarios 

These  scenarios  should  incorporate  as  near  real 
conditions  as  feasible.  Needless  to  say,  as  in  any 
training  situation,  the  environment  should  ini¬ 
tially  be  somewhat  simplistic:  for  example,  one-on- 
one.  Then,  as  proficiency  grows,  the  scenarios 
should  become  increasingly  complex  until  the  air¬ 
crews  are  able  to  cope  with  a  maze  of  defensive 
threats  and  distractions  and  still  achieve  their 
primary  objective.  Surprise  (encounter  with 
unanticipated  threats)  should  be  inherent  in  such  a 
scenario.  Implicit  in  providing  realism  should  be 
the  effect  of  terrain.  Since  terrain  masking  must 
be  considered  one  of  the  most  effective  ways  of 
evading  or  defeating  a  threat,  aircrews  should  be 
trained  to  use  terrain,  taking  it  into  account  dur¬ 
ing  mission  preparation  and  instinctively  including 
it  in  reactive  options  during  flight. 


•  REALISTIC  SCENARIO 

*  INTEGRATED  OFFEN S I VE rtJEFEN S l VE 

*  VARYING  DENSITY 

•  CREW  INTERACTION 

*  RECOGNIZE,  ASSESS,  ACT 

*  REAL-TIME  FEEDBACK 

*  NO  NEGATIVE  TRAINING 

•  CONTINUATION  TRAINING 

*  FREQUENT 

*  COUPLED  TO  COMBAT  TRAINING  MISSIONS 


FIGURE  4.  TRAINING  REQUIREMENTS 

Crew  Interaction 

Crawa  must  be  able  to  exsrtise  prescribed  doc¬ 
trine,  be  it  meneuvera  or  employment  of  onboard 
countermeasurea  systems,  so  as  to  defeat  threats. 
When  actions  are  correct  and  timely,  crawe  must 
receive  feedback  in  terms  of  negation  of  the  threat . 
When  inappropriate ,  they  should  receive  indications 
of  failure,  such  as  becoming  simulated  casualties. 
And  most  important,  they  ahould  not  receive  negative 
training.  They  should  not  experience  success  if 
they  ignore  or  react  incorrectly.  Conversely,  they 
should  not  continue  to  be  subjected  to  a  threat  if 
they  follow  the  correct  procedure. 

While  it  is  true  that  in  real  life  exercise  of 
proper  procedures  may  not  always  guarantee  success, 
it  is  most  important  that  the  training  system  not 
instill  in  aircrews  such  subjective  scepticism  that 
they  not  try.  For  example,  if  a  simple  emitter  were 
put  on  a  route  and  it  were  to  activate  the  aircraft 
radar  warning  system,  and  if  the  aircrew  were  then 
unable  to  defeat  it  by  employing  their  counter¬ 
measures  in  conjunction  with  prescribed  maneuvers, 
they  might  eventually  learn  to  simply  ignore  it. 
Since  following  prescribed  procedures  would  add  to 
the  difficulty  of  concentrating  on  their  navigation 
under  such  conditions  but  would  do  nothing  to 
increase  their  perceived  safety,  they  would  be 
tempted  to  do  the  wrong  thing. 

To  acclimate  crews  properly,  they  should  have 
to  interact  with  defensive  scenarios  every  time  they 
fly  combat  training  missions  so  as  to  be  psychologi¬ 
cally  prepared  each  time  they  start  planning  a  com¬ 
bat  sortie. 


Continuation  Training 

Since  learning  procedures  that  must  be  imple¬ 
mented  almost  as  an  instinctive  reaction  requires 
constant  iteration  and  practice,  it  is  most 
important  that  the  prescribed  readiness  criteria 
entail  frequent,  continual  training  of  this  sort. 
No  one  would  suggest  that  a  Navy  pilot  could  shoot 
10  carrier  landings  in  a  2-week  period,  once  a  year, 
and  still  be  proficient  9  months  later  —  not  unless 
he  were  practicing  the  equivalent  on  a  continuing 
basis  throughout  that  period. 
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Sine*  most  unit*  do  not  have  access  to  major 
hardware  ranges  on  a  continuing  basis,  routina  local 
training  today,  particularly  (or  tactical  air  (orcaa 
atationod  in  Buropa  and  tha  Far  Eaat,  doaa  not  par* 
ait  intogratad  of fensive/defensive  training  aa 
deacribad  abova. 


PHANTOM  RANGE 

The  Phantom  Ranga,  which  provides  (or  the 
desired  training  on  a  continuing  basis  at  local 
installations,  consists  primarily  of  a  pod  -- 
Phantom  Range  Pod  (PRP)  —  carried  on  a  fighter  air¬ 
craft  standard  wing  pylon  (Figure  5).  Ground 
support  hardware  for  the  system  consists  of  a 
Mission  Planning  and  Debriefing  Station  (MPDS)  and  a 
Flightline  Support  Unit  (FSU). 


PRP 

Inside  the  PRP  (a  6-ft-long  by  8-in-diameter 
pod)  is  a  navigation  system  consisting  of  a  Loren 
receiver  and  a  strapdown  inertial  unit  (Figure  6). 


FIGURE  5.  PHANTOM  RANGE  APPROACH 


FIGURE  6.  PRP  -  A  DERIVATIVE  OF  TRIPOO 


This  navigation  system  is  capable  of  locating  the 
aircraft  position  at  all  times,  even  during  and 
after  hard  maneuvers. 

The  PRP  also  contains  a  minicomputer  within 
whose  memory  it  stored  the  geographic  location  of 
various  simulated  threats  and  the  lethal  envelopes 
modified  by  actual  terrain  surrounding  those  loca¬ 
tions  (Figure  7).  Additionally,  in  memory  are 
stored  threat  defeat  procedures,  be  they  maneuvers 
or  switch  positions  of  countermeasures  equipment. 

When  the  airersft  penetrates  the  lethal 
envelope  at  discerned  by  the  Loren  (Figure  8),  the 
computer,  having  computed  azimuth  and  range,  com¬ 
mands  the  radar  warning  system  to  display  the  proper 
video  signal  on  the  radar  warning  indicator.  It 
also  lights  the  appropriate  lights  on  the  control 
panel  and  sends  the  proper  audio  signal  through  the 
intercom  system.  If  communications  jaasaing  is 
desired,  noise  can  be  generated  by  the  audio 
generator,  which  is  used  to  construct  synthetic 
radar  warning  audio  signals  and  which  can  be 
injected  concurrently  into  the  intercom.  (While  the 
system  as  currently  conceived  does  not  provide  jam¬ 
ming  of  the  aircraft  radar  or  navigation  system, 
this  capability  could  be  added  if  desired.) 


FIGURE  7.  FUNCTIONAL  SUBASSEMBLIES  IN  THE  PRP 
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The  computer  alio  infer*  a  launch  after  a 
nominal  period  subsequent  to  track  radar  lock-on. 
Depending  on  distance  from  launch  site  to  aircraft , 
missile  fly-out  time  is  computed,  and  if  the  threat 
has  not  been  defeated  within  that  time,  the  video 
flashes  and  a  buster  sounds  in  the  intercom  to  indi¬ 
cate  the  aircraft  has  been  hit.  After  1  sec,  the 
system  restates  normal  operation  so  that  the  exercise 
can  continue, 

Finally,  all  visual  and  audio  cue*  are 
recorded,  as  are  significant  aircraft  and  relevant 
aircrew  threat  defeat  actions.  These  can  then  be 
inserted  in  the  MPDS  for  crew  debriefing. 


MPDS 

The  Mission  Planning  Debriefing  Station 
(Figure  9)  has  two  functions!  1)  preparing  a  RAM 
mission  module  for  insertion  into  and  control  of  the 
PRP  during  flight  and  2)  reconstructing  the  mission 
for  debriefing. 

For  mission  preparation,  the  MPDS,  consisting 
of  a  minicomputer  and  input/output  devices,  receives 
generic  threat  data!  i.e.,  lethal  and  audio  charac¬ 
teristics,  envelopes,  geographic  positions  of  each 
threat,  significant  terrain  features  surrounding 
each  threat,  and  prescribed  threat  defeat 
procedures.  These  are  inputted  by  EPROM  in  the  case 
of  generic  threat  data  or  threat  defeat  procedures, 
and  either  by  EPROM  or  manually  in  the  case  of  posi¬ 
tion  location  and  terrain.  Thus,  if  positions  of 
several  threats  are  to  be  changed,  the  operator  need 
only  delete  the  original  locations  and  surrounding 
terrain  and  insert  new  locations  of  such  threats  and 
the  terrain  surrounding  the  new  locations.  An 
alphanumeric  display  is  used  to  call  up  information 
from  the  computer  and  display  to  the  operator 
results  of  his  normal  input.  A  atrip  printer  pro¬ 
vides  hardcopy  of  relevant  data,  such  as  threat 
locations . 

Once  the  computer  has  received  inputs,  it  pre¬ 
pares  the  mission  module  so  that  it  contains  the 
appropriate  threat  and  threat  defeat  data.  Then  the 
mission  module  is  removed  from  the  MPDS  and  inserted 
into  the  PRP  on  the  flightline. 

For  the  debriefing  portion,  a  digital  tape  is 
removed  from  the  PRP  and  inserted  into  the  MPDS. 
Data  on  the  tape  are  compressed  for  flight  activi¬ 


ties  unrelated  to  the  EW  portion  of  the  mission, 
while  relevant  data  are  sampled  every  1  aec.  The 
displays  (the  Radar  Warning  Scop*  and  Control  Panel 
plus  a  speaker  for  audio)  are  activated  as  they  were 
in  flight,  while  other  significant  parameters  are 
shown  on  the  alphanumeric  display  (Figure  10). 


FSU 

The  Flightline  Support  Unit  (Figure  11)  serves 
to  preflight  the  PRP,  initialise  the  Loran,  check 
the  input  of  the  mission  module,  and  provide  a  go, 
no-go  check.  It  also  serves  to  diagnose  end  isolets 
faults  or  malfunctions  to  the  Line-Replaceable-Unit 
level. 

The  FSU  consists  of  a  microprocessor,  an 
alphanumeric  display,  and  a  keyboard  for  operator 
control . 


PRP/Aircraft  Interface 

The  PRP  uses  aircraft  wiring  currently  existing 
at  wing  stations  capable  of  carrying  a  jamming  pod 
(Figure  12).  It  mounts  on  a  standard  stores  pylon. 

Electrical  power  (28  Vdc)  existing  at  this 
station  is  used  to  power  the  PRP,  with  the  PRP 


TIM1  . .  . 

AIRCRAFT  LATITUDE  LONGITUDE  ALTITUDE  MOO  A/I  III 

ECM  SWITCH  POSITION  CHAFF  SWITCH  POSITION 
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FIGURE  10.  PHANTOM  RANGE  DISPLAY  INFORMATION 


FIGURE  9.  MISSION  PLANNING  DEBRIEFING  STATION 
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FIGURE  11.  FLIGHTLINE  SUPPORT  UNIT 


\ 


FIGURE  12.  F-4E  ECM  CABLE  HARNESS 


operating  any  tln«  powar  ia  available  at  the  pylon. 
(A  email  rechargeable  battery  in  the  PRP  maintain# 
volatile  memory  when  aircraft  powar  ie  off  or  if 
there  ia  a  momentary  interrupt.) 

Commands  for  the  video  dieplaya  are  tent 
through  the  radar  warning  system,  which  in  turn 
generate#  the  appropriate  aymbology  and  light# 
( figure  13).  Audio  ia  generated  aynthetically  aince 
no  actual  PRF  ia  available.  The  audio  ia  then 
carried  to  the  cockpit  over  exiating  wires  and 
inaerted  into  the  intercom  ayatem  (ICS)  by  means  of 
a  email  jumper  cable  (Figure  14).  The  aynthetic 
audio  generator  alao  createa  noiae  representative  of 
coiaaunications  jamming  and  inserts  this  into  the  ICS 
in  a  similar  manner. 

By  uae  of  such  an  interface,  the  operational 
radar  warning  ayatem  can  function  normally, 
displaying  real  threats  detected  by  the  receiver 
a imultaneoualy  with  Phantom  Range  threats.  Thus  the 
aircraft  remains  operationally  ready  at  all  times. 

Finally,  pilot  action  discrete  signals,  such  as 
chaff  deploy  and  jammer  switch  positions,  are 
monitored  at  the  pylon  and  introduced  into  the 
processor.  These,  together  with  aircraft  maneuvers, 
provide  the  basis  for  determining  whether  the 
aircrew  has  performed  the  proper  threat  defeat  pro¬ 
cedures  within  an  allowable  time. 


TRAINING  UTILITY 

The  Phantom  Range  would  be  located  at  wing 
level  with  10  to  12  PRPs,  2  MPDSa,  and  2  FSUa  per 
wing.  Mission  modules  would  be  prepared  locating 
threats  along  locally  accessible,  low-level  routes 
used  as  ingress  corridors  to  ordnance  delivery 
ranges  or  for  photo-reconnaissance  missions.  A 
number  of  such  modules  of  varying  complexity  would 
be  prepared  ranging  from  one  threat  at  a  time  and  no 
comunicationa  jamming  to  a  thicket  of  up  to  10 
threats  displayed  simultaneously  and  heavy 
communications  jamming.  As  aircrews  became  more 
proficient,  they  would  fly  more  and  more  complex 
scenarios  until  they  are  able  to  cope  with  each 
situation  that  would  have  been  accurately  pre¬ 
briefed  by  intelligence  and  planned  for  by  the 
mission  leader. 

At  this  point,  threat  locations  could  be 
altered  such  that  the  information  received  in  the 


FIGURE  13.  ACTUAL  RWR  EQUIPMENT  DRIVEN  BY  THF 
PHANTOM  RANGE  EQUIPMENT 


FIGURE  14.  JUMPER  CABLE  PROVIDING  AUDIO  INTERFACE 


intelligence  briefing  would  become  more  and  more 
unreliable  and  threats  would  pop  up  where  least 
expected.  The  ultimate  condition  would  be  for  the 
aircrew  to  fly  a  mission  in  which  all  threats  appear 
at  locations  other  than  those  briefed  or  where  no 
expected  location  information  was  provided  prior  to 
flight. 

Aircrews  would  be  forced  to  take  hard  evasive 
action,  deviating  from  and  reacquiring  planned 
routes,  while  at  the  saa»e  time  finding  and  turning 
switches  on  and  off  as  appropriate.  (Jammers  would 
be  turned  off  as  soon  aa  a  lock  was  broken  so  as  not 
to  allow  the  simulated  threat  to  reacquire  through 
an  implicit  home-on- jam  feature.) 

Once  aircrews  had  achieved  and  were  able  to 
maintain  requisite  proficiency,  they  would  be 
prepared  to  engage  in  periodic  Red  Flag  exercises 
where  ground  personnel  operating  realistic  threat 
emulators  would  be  able  to  interact  with  the 
aircraft,  fire  simulated  smoke  rockets,  and  provide 
the  ultimate  in  realism. 


CONCLUSIONS 

The  Phantom  Range  permits  continuous  training 
in  all  normal  tactical  air  force  flying  areas, 
independent  of  ranges  or  ground  emitters;  it  permits 
the  aircraft  to  remain  operationally  ready  at  all 
times;  it  provides  realistic,  increasingly  complex 
scenarios;  it  enables  the  aircrews  to  interact  on  a 
real-time  basis  in  flight;  and  it  furnishes  the 
capability  for  reconstruction  during  debriefing. 
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By  ■•fciit  of  frequent  local  training  using 
Phantoa  Rang*,  aircrawa  can  b«  exercised  in  all 
phase*  of  alsslon  planning,  flight  operation*  and 
tactic*,  and  datailad  debriefing  ao  aa  to  aaintain 
th*  daalrad  etat*  of  proficiency  at  all  tiaee.  They 
can  laarn  to  react  inetlnctively  to  near-real-world 
condition*  of  high  etraaa  load*  and  eurpriaa. 
Should  war  occur,  they  would  not  have  to  endure  th* 
loeae*  aaeociated  with  on-the-job  training  during 
th*  firat  10  coabat  alaalona,  aa  experienced  in 
Vietnaa. 
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ABSTRACT 


This  paper  introduces  a  candidate  model  for  development  of  wargame-based  training  systems 
and  describes  the  application  of  that  concept  during  development  of  the  Naval  Tactical  Game 
(NAVTAG)  Training  System.  A  wargame-based  training  system  combines  the  inherent  educational 
advantages  of  simulation  and  gaming  with  the  best  features  of  more  formal  or  conventional  meth¬ 
ods  of  instruction. 

Research  indicates  that  gaming  simulation  has  the  potential  to  afford  significant  advan¬ 
tages  over  conventional  classroom  methods  of  training.  This  appears  to  be  particularly  true 
for  complex  skills  such  as  goal-oriented  decision-making.  Wargames  can  be  made  more  effective 
by  using  an  integrated  development  approach  based  upon  the  systems  approach  to  instructional 
development,  and  the  addition  of  a  complete  set  of  courseware. 

Some  key  features,  or  characteristics,  of  wargame-based  training  systems  are  that  they: 

•  Utilize  an  Instructional  Systems  Development  Approach 

•  Provide  a  Complete  Wargame  Training  Package 

•  Are  Usable  in  Game  or  Nongame  Modes 

•  Are  Scenario- Independent 

•  Provide  Realistic  Threat  Portrayal 

•  Provide  Basic  and  Advanced  Modes 

•  Are  Easily  Modified 


Other  design  features  include  a  reasonable  level  of  complexity,  realism  and  playability; 
and  a  requirement  for  a  minimum  number  of  support  personnel. 

Wargame-based  training  systems  are  a  unique  approach  to  the  special  challenges  of  tactical 
decision-making  training.  The  concept  is  applicable  to  all  services  and  can  easily  be  expanded 
to  accommodate  the  introduction  of  new  technologies,  weapons  systems  and  tactics. 

The  authors  illustrate  how  the  design  and  development  of  a  wargame-based  training  system 
is  accomplished  by  reference  to  NAVTAG  (Naval  Tactical  Game).  NAVTAG  will  be  used  aboard  ship, 
probably  in  the  wardroom,  to  afford  officers  an  opportunity  to  enhance  through  practice  (or  to 
learn,  then  practice)  skills  associated  with  tactics  and  tactical  decision-making.  The  NAVTAG 
Training  System  is  intended  to  complement  other  methods  of  tactical  and  team  training,  but 
will  not  attempt  to  simulate  the  physical  environment  or  provide  a  basis  for  team  training. 
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INTRODUCTION 

Tactical  training  was  an  early  casualty  of  the 
restrictive  budgets  and  decreased  operational  tempo 
of  this  past  decade.  As  valuable  operational  train¬ 
ing  time  declined,  the  initial  tendency  was  to  com¬ 
pensate  through  Increased  use  of  real-time  simula¬ 
tion  techniques  for  team  and  subteam  training;  with 
tactical  training  an  adjunct  to  other  types  of 
training.  In  the  face  of  accelerating  technological 
advances  and  introduction  of  new  systems,  tactical 
training  Itself  has  become  more  complex  and,  there¬ 
fore,  increasingly  more  expensive.  Wargaming  has 
been  recognized  as  one  way  in  which  to  complement 
and  enhance  the  overall  value  of  tactical  decision¬ 
making  training.  In  fact,  wargames  have  been  used 
for  training,  in  a  variety  of  forms,  fora  long  time. 

Published  literature  contains  many  articles  in 
the  area  of  instructional  systems  development  and 
the  use  of  simulation  and  games  for  training  pur¬ 
poses.  There  is,  however,  very  little  in  the  lit¬ 
erature  on  the  development  of  wargames  to  be  used 
for  training.  Recent  developments  and  growing  in¬ 
terest  in  the  use  of  wargames  for  tactical  training 
applications  have  underscored  the  need  for  a  model 
for  development  of  wargames  and  related  instruc¬ 
tional  materials.  This  paper  presents  a  candidate 
model  for  development  and  implementation  of  wargaroe- 
based  training  systems,  within  the  general  frame¬ 
work  of  the  systems  approach  to  development  of 
instruction.  For  our  purposes  here,  a  wargame- 
based  training  system  is  defined  as  an  instruction¬ 
al  system  which  has  a  wargame  as  an  integral  part. 
The  wargame  may  be  used  as  instructional  media  re¬ 
quiring  direct  interface  with  students,  or  as  sup¬ 
port  for  a  simulation  exercise,  based  on  indirect 
interface  with  students. 


BACKGROUND 

Wargames  combine  the  elements  of  both  simula¬ 
tion  and  games  in  a  military  context.  Simulations 
involve  the  imitation  of  real-life  systems  or  oper¬ 
ations  through  the  use  of  a  variety  of  techniques 
including  terrain  boards,  computers  and  other  aids. 
Games  involve  some  element  or  level  of  competition 
among  individuals  for  the  purpose  of  achieving  cer¬ 
tain  pre-specif ied  goals.  Wargames  bring  together 
the  two  concepts  of  simulation  and  games  in  appli¬ 
cations  that  include  operational  planning,  analysis, 
evaluation,  and  training.  Wargames  used  for  train¬ 
ing  can  be  of  several  basic  types,  including  manual 
wargames  and  computer-assisted  or  automated  war- 
games.  (1) 

Conflict  simulations  have  long  been  used  to 
prepare  for  combat.  One  of  the  better-known 
accounts  of  the  use  of  wargames  is  that  of  Japanese 
preparations  for  the  Battle  of  Midway.  In  this 
elaborate  game,  Japanese  naval  officers  plaving  the 
role  of  Americans  launched  an  attack  on  the  Japa¬ 
nese  carrier  force, inflicting  devastating  losses. 
When  two  of  the  Japanese  carriers  were  sunk,  Admiral 
Ugaki  objected  to  the  umpires'  ruling,  and  the  car¬ 
riers  were  declared  safe.  In  effect,  the  two  car¬ 
riers  were  "refloated."  The  game  then  went  on  to 
indicate  the  victory  at  Midway  that  senior  Japanese 
officers  felt  was  inevitable.  During  the  real  bat¬ 
tle,  the  Japanese  carrier  force  was  struck  almost 
precisely  as  indicated  by  the  earlier  wargame,  but 
with  even  more  disastrous  results  for  the  Japanese, 
as  all  four  carriers  were  lost.  (2) 


The  development  of  computers  has  added  another 
dimension  to  the  design  and  use  of  wargames.  Our 
Defense  and  State  Departments  make  extensive  use  of 
computer  wargames  to  plan  and  analyze  policy  deci¬ 
sions  and  options.  Foreign  policy  crises  have  also 
been  gamed  by  civilian  and  military  planners.  A 
recent  example  of  this  application  of  wargaming 
was  the  use  of  gaming  by  both  military  and  politi¬ 
cal  planners  prior  to  the  Isreali  raid  on  Entebbe. 
Today  manual  and  automated  wargames  are  in  use  by 
all  the  services.  The  cost/benefit  advantages  of 
games  for  training  are  becoming  increasingly  apar- 
ent  as  instructional  designers  become  more  familiar 
with  the  medium  and  its  applications;  often  in  what 
might  be  considered  as  new  or  nontradltional  set¬ 
tings  . 


TRAINING  WITH  WARGAMES 

An  understanding  of  some  of  the  characteristic 
features  of  wargames  is  important  to  an  understand¬ 
ing  of  their  use  for  training. 

Wargames  put  players  in  situations  that  model 
real  world  systems  or  portions  of  systems.  The  in¬ 
tention  is  not  to  completely  duplicate  reality,  but 
to  provide  experiences  which  are  transferable  to  an 
actual  operational  problem.  The  important  point  is 
that  wargames  are  Intentionally  not  totally  realis¬ 
tic,  at  least  in  terms  of  physical  realism.  Just 
as  we  would  not  want  to  rely  upon  a  wargame  to  spe¬ 
cifically  predict  the  outcome  of  future  conflicts 
(primarily  because  of  the  number  of  variables  mod¬ 
eled  and  the  assumptions  made),  we  also  would  not 
want  our  wargame  player-students  to  assume  that 
their  performance  in  a  game,  and  a  specific  outcome, 
are  totally  transferable  to  the  real  world.  There¬ 
fore,  an  understanding  of  the  artificialities  im¬ 
posed  by  the  wargame  is  essential  to  successful 
transfer  from  the  game  to  the  real-world  operation¬ 
al  setting. 

Wargames  may  involve  only  one  player  or  they 
may  involve  groups  of  many  players .  The  players 
strive  to  achieve  certain  goals  through 
their  participation.  At  the  same  time,  players  are 
constrained  by  the  scenario  and  operating  proce¬ 
dures.  Successful  play  sometimes  involves  agree¬ 
ment  to  abide  by  conditions  which  may  not  exist  in 
the  real  world.  For  example,  time  may  be  expanded 
or  compressed  during  play  of  the  game.  In  addition, 
players  may  be  assigned  roles  in  the  game  which  do 
not  necessarily  conform  to  those  assumed  in  real 
life.  Regardless  of  other  characteristics,  most 
wargames  include  some  method  to  record  events, 
decisions  and  results;  and  to  determine  winners. 

In  the  past,  wargames  often  have  been  viewed  as 
valuable  analytical  tools,  but  lacking  in  suffici¬ 
ent  effectiveness  for  serious  training  applications. 
It  is  the  authors'  contention  that  this  view  is  not 
supported  by  recent  experience  with  the  use  of  war- 
games  by  the  various  services. 

WARGAME-SPECIFIC  ISSUES 

Although  answering  questions  about  the  effect¬ 
iveness  of  wargaming  as  an  instructional  medium  is 
outside  the  scope  of  this  paper,  a  brief  discussion 
of  the  advantages  and  limitations  of  wargames  may 
help  to  set  the  context  for  the  discussion  of  the 
development  of  wargame-based  training  systems  that 
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follows. 


Wargames  used  for  training  exhibit  advantages 
that  may  be  considered  In  two  broad  categories: 

(1)  enhancement  of  the  learning  environment;  and 

(2)  Improvements  In  Instructional  management. 

Instructional  wargames  provide  opportunities 
for  students  to  experience,  in  a  controlled  environ¬ 
ment,  the  consequences  of  decisions  which  may  be 
encountered  in  an  operational  setting.  A  wargaroe 
can  provide  the  valuable  experiences  of  learning 
through  participation  that  otherwise  could  be  ob¬ 
tained  only  through  real-time,  full-scale  simula¬ 
tions,  field  exercises,  or  actual  combat  operations. 
Wargames  can  help  avoid/reduce  the  costs  and  diffi¬ 
culties  often  associated  with  full-scale  simula¬ 
tions,  exercises  and  operations,  while  providing 
much  of  the  training  value  of  the  more  expensive 
alternati ves . 

Proponents  of  games  or  wargames  for  instruc¬ 
tion  point  to  advantages  of  increased  student  moti¬ 
vation.  Actually,  little  is  known  about  the  moti¬ 
vational  power  of  games  beyond  the  indications  that 
the  elements  of  chance,  competition  and  excitement 
tend  to  stimulate  learning.  (3)  There  is  little 
disagreement  that,  at  a  minimum,  these  elements 
stimulate  participation  and,  therefore,  can 
enhance  the  opportunities  and  potential  for  learn¬ 
ing  . 

Because  wargames  establish  situations  that 
evolve  with  student  interactions,  they  seem  to  be 
an  effective  way  to  teach  the  structure  and  opera¬ 
tion  of  systems.  (A)  Decision-making  skills  can  be 
strengthened  through  opportunities  to  see  the 
results  of  a  series  of  decisions  unfold.  The  avail¬ 
ability  of  such  results  in  the  form  of  near  real¬ 
time  feedback  is  an  advantage  of  wargaming  that 
does  not  always  exist  in  other  instructional  media; 
even  including  field  exercises. 

In  addition  to  the  potential  for  enhancements 
to  the  learning  environment,  the  use  of  wargames 
for  instructional  purposes  has  physical  and  logis¬ 
tic  support  advantages  for  instructional  management. 
Wargaraes  can  be  designed  to  be  both  exportable  and 
transportable.  In  fact,  the  entire  instructional 
system  package  can  be  made  relatively  small  and 
light-weight  for  ease  of  transportation  in  the 
field  or  aboard  ship.  Small  packaging  also  can 
ease  maintenance  and  repair  support  requirements. 

In  certain  applications,  wargaming  can  be  more 
cost-effective  than  other  media.  The  potential  to 
bring  the  instructional  package  to  the  individual, 
and  to  require  minimal  instructor  resources,  are 
examples  of  the  cost  benefits  of  wargames. 

Wargaming  complements  other  forms  of  decision¬ 
making  or  tactical  training.  A  wargame  can  provide 
opportunities  to  review  tactical  experiences  or  to 
prepare  for  future  exercises.  Areas  of  individual 
weakness  in  prerequisite  knowledge  can  be  easily 
identified  in  a  wargame  environment.  Wargames  may 
also  be  tailored  to  specific  needs;  such  as  time 
available  for  training.  Thus,  wargames  can  provide 
for  regular  proficiency  and  refresher  training  for 
tactical  decision-makers;  working  easily  around 
other  demands  on  an  individual's  or  group's  time. 

Although  there  are  numerous  advantages,  there 
are  also  limitations  to  the  use  of  wargames  for 
training.  Research  indicates  that  Instructional 


simulations  are  successful,  but  not  necessarily 
superior  to,  more  conventional  techniques  for  teach¬ 
ing  facts  or  bodies  of  knowledge.  (5)  Simulation 
requirements  also  can  be  expensive  and,  therefore, 
cost-prohibitive  in  certain  applications. 

Wargames  can  also  be  time-consuming  and  diffi¬ 
cult  to  learn  how  to  play.  While  this  limitation 
has  in  large  measure  been  overcome  by  computer  auto¬ 
mation,  automation  has  its  own  limitations.  For 
instance,  computer  automation  is  expensive  and  can 
introduce  additional  artificialities  and  obstacles 
to  transfer  of  training. 

Unless  a  wargame  is  carefully  constructed  and 
monitored  as  part  of  an  instructional  system,  it  may 
have  a  tendency  to  lead  student  players  to  develop 
unrealistic  attitudes  toward  the  real  world  opera¬ 
tional  environment  or  their  own  abilities.  Of  re¬ 
lated  concern  is  the  fact  that  the  outcomes  of  mod¬ 
els  incorporated  in  a  wargame  rarely  can  be  fully 
validated.  Empirical  validation  of  models  through  a 
"preponderance  of  evidence"  is  perhaps  the  best 
means  available  to  validate  wargame  models. 

In  addition  to  the  potential  for  development  of 
unrealistic  attitudes,  or  expectations,  there  exists 
with  wargames  the  potential  that  students  will  use 
the  game  to  reinforce  existing  biases.  The  student 
thus  may  bring  to  the  game  a  "favorite"  solution,  re¬ 
gardless  of  the  nature  of  the  problem  or  may  tend  to 
view  all  problems  as  being  essentially  identical. 

(6)  Kapper  has  noted  that  "the  most  blatant  abuse 
(of  wargames)  today. ..is  advocacy."  (7)  This  in¬ 
cludes  the  use  of  wargames  to  "sell"  programs  where 
vested  interests  are  at  stake,  or  to  denigrate  pro¬ 
grams  over  which  there  is  a  disagreement  or  uncer¬ 
tainty.  An  example  of  this  could  be  the  justifica¬ 
tion  of  a  new  tactical  doctrine  through  the  "re¬ 
sults"  of  the  wargame. 

The  possibility  that  these  limitations  and 
potential  disadvantages  will  have  negative  impacts 
on  training  can  be  lessened  through  a  systems 
approach  to  development  of  wargame-based  training, 
systems . 


IMPORTANCE  OF  A  SYSTEMS  APPROACH 

The  development  of  instructional  systems  has 
long  been  recognized  as  a  complex  process.  Systems 
engineering  accounts  for  and  demands  a  reasoned 
transition  from  precise  needs  statements,  to  vali¬ 
dation  of  problem  environment  and  situation, 
through  identification  of  all  alternative  solution 
candidates  and  optimal  selection,  implementation 
and  continuing  refinement.  The  instructional  sys¬ 
tems  development  procedures  followed  by  the  services 
are,  in  fact,  the  application  of  systems  engineering 
principles  to  the  problems  of  training  systems  devel¬ 
opment.  The  application  of  these  procedures  repre¬ 
sents  the  transition  from  art  to  quasi-science  in 
the  development  of  instructional  systems.  Vlhen  the 
problems  of  the  unique  characteristics  and  varied 
implementation  philosophies  of  wargame-based  train¬ 
ing  systems  were  confronted,  it  was  logical  that  the 
instructional  systems  development  model  be  utilized 
as  the  basis  for  development  of  a  tailored  model. 

One  of  the  challenges  in  the  adaptation  of  the 
instructional  systems  development  model  is  that  the 
media  selection  process  often  has  been  obviated  by 
decisions  which  preceded  analysis  of  the  training 
needs.  Validation  of  the  results  of  wargame 
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exercises  present  yet  another  challenge.  This  situ¬ 
ation  is  further  exacerbated  by  the  paucity  of  real 
data  to  validate  model  and  strategy  effectiveness. 
For  these  reasons,  considerable  effort  has  gone  in¬ 
to  ensuring  significant  subject  matter  expert  inputs 
and  reviews  during  the  development  process.  Further, 
because  of  the  additional  impact  of  new  weapons/ 
sensors  technology,  much  more  emphasis  has  been 
placed  on  the  evaluation  and  management  phases  after 
implementation  of  the  wargame-based  training  system. 
The  known  complexity  and  problems  in  the  development 
of  cost/training  effective  and  efficient  instruc¬ 
tional  systems,  when  coupled  with  the  unique  prob¬ 
lems  and  characteristics  of  wargaming,  underscore 
the  requirement  that  the  technology  of  instructional 
systems  development  be  adapted  to  produce  an  opera¬ 
tional  model  for  development  of  wargame-based 
training  systems. 


The  authors  have  developed  a  candidate  model 
for  development  of  wargame-based  training  systems. 
This  model  Includes  appropriate  parts  of  the  model 
for  the  systems  approach  to  development  of  training 
materials.  As  currently  configured,  there  are  12 
steps  in  the  model,  as  discussed  in  the  following 
section. 


DEVELOPMENT  OF  WARGAME-BASED  TRAINING  SYSTEMS 

Figure  1  illustrates  the  12  steps  in  the 
development  of  wargame-based  training  systems.  Al¬ 
though  the  model  is  portrayed  in  a  linear  fashion, 
it  is  not  a  requirement  that  each  step  be  conducted 
sequentially.  Some  activities  may  be  accomplished 
concurrently,  or  even  out  of  sequence. 


STEPS  IN  DEVELOPMENT  OF  A  WARGAME-BASED  TRAINING  SYSTEM 


LEGEND 


DEVELOPMENT 

PATH 


FEEDBACK 

PATH 


Figure  1.  Steps  in  Development  of  a  Wargame-Based  Training  System 


Discussion  of  the  model  will  focus  on  the  ac¬ 
tivities  to  be  accomplished  during  each  step,  and 
the  products  which  will  result  from  those  activi¬ 
ties.  Where  the  racdel  closely  follows  the  instruc¬ 
tional  systems  development  model,  little  informa¬ 
tion  will  be  presented,  as  this  information  is 
available  elsewhere.  Instead,  unique  activities 
and/or  products  of  this  model  will  be  described  in 
some  detail.  Considerations  for  development  of 
automated  systems  are  also  presented. 


Step  1.0  involves  the  analysis  of  the  training 
problem,  and  is  similar  to  the  first  two  steps  out¬ 
lined  in  MIL-T-29053B  (Military  Specification: 
Requirements  for  Training  System  Development).  The 
primary  products  of  this  step  are  a  Problem  Analysis 
Report  and  a  Iraining  System  Development  Plan, 
which  together  specify  the  approach  required  for 
development  and  implementation  of  the  training 
system. 
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Step  2.0,  training  analysis  ar.d  design,  in¬ 
volves  the  analysis  activities  included  in  Phases  I 
and  TI  of  the  instructional  systems  development 
model . 

The  primary  products  of  Steps  1.0  and  2.0, 
then,  should  be  a  list  of  tasks  selected  for  train¬ 
ing  and  a  list  of  hierarchial,  or  sequenced  learn¬ 
ing  objectives.  These  become  primary  inputs  to 
Step  3.0,  design  of  the  wargame-based  training 
system. 

Although  not  unique  to  the  development  of  this 
type  of  training  system,  it  is  important  for  the 
reader  to  recognize  that,  given  the  exigencies  of 
operating  in  the  "real-world,"  It  is  possible  that 
some  decisions  -  including  media  selection  -  may  be 
used  in  place  of  original  research.  If  the  activi¬ 
ties  required  in  Steps  1.0  and  2.0  have  been  com¬ 
pleted  (o  the  need  for  such  activities  obviated  by 


decisions  imposed  on  the  use  of  the  model)  the  de¬ 
velopment  of  a  wargame-based  training  system  may 
begin  with  Step  3.0.  A  considerable  stress,  how¬ 
ever,  will  be  placed  on  the  development  process 
(and  on  the  developers  of  the  system),  since  the 
results  of  the  decision,  or  lack  of  sufficient  an¬ 
alysis,  can  create  problems  later  in  the  process. 


Design  Waraame-Based  Training 


The  activities  that  take  place  during  Step  3.0 
are  illustrated  in  Figure  2.  Activities  3.1  and 
3.2  involve  the  specification  of  learning  activi¬ 
ties  (emphasizing  the  type  and  methods  of  feedback 
to  be  provided  to  students)  and  specification  of 
the  instructional  management  plan.  Through  these 
activities,  the  instructional  developer  should  look 
to  verify  the  selection  and  appropriateness  of  the 
wargame  medium  and  to  determine  how  the  instruc¬ 
tional  system  is  to  be  packaged  and  implemented. 


STEP  3.0  -  DESIGN  THE  TRAINING  SVSTEM 


Figure  2.  Step  3.0  -  Design  the  Wargame-Based  Training  System. 


Activity  3.3  focuses  on  the  wargame,  itself. 

The  functional  characteristics  of  the  wargane  (and 
the  total  system)  should  reflect  the  results  of 
many  decisions,  all  made  in  the  context  of  the 
learning  objectives.  Some  of  these  decisions  in¬ 
clude  resolution  of  these  issues: 

•  Is  the  game  to  be  used  for  practice  only,  or 
will  it  be  used  as  an  instructional  delivery 
device? 

•  Should  the  game  be  manual,  computer-assisted 
or  fully  automated? 

•  Should  the  wargame  be  free-play  or  scripted? 

•  Should  the  wargame  be  one-sided  or  two-sided? 


•  Should  the  game  be  in  the  open  or  closed  mode? 

The  game's  functional  characteristics,  instruc¬ 
tional  management  plan  and  learning  objectives  will 
serve  as  the  primary  inputs  to  the  design  of  the 
training  system.  The  remaining  steps  involve  the 
design  of  various  subsystems  of  the  training  sys¬ 
tems.  The  subsystems  of  a  wargame-based  training 
system,  illustrated  in  Figure  3,  include: 

•  Instructional  Subsystem 

•  Simulation/ Game  Subsystem 

•  Critique/Feedback  Subsystem 

•  Implementation  Subsystem 
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COMPONENTS  OF  SUBSYSTEMS 


Figure  3.  Wargame-Based  Training  Subsystems  and  Components 


Activity  3.4,  design  of  the  Instructional  Sub¬ 
system,  results  in  the  curriculum  outline,  lesson 
plan  and  scenario  requirements,  and  design  criteria 
for  the  Implementation  Subsystem. 

The  specification  of  the  simulation/gaming 
requirements  (Activity  3.5)  results  in  design  cri¬ 
teria  for  the  wargame,  using  as  primary  inputs  the 
wargame  functional  characteristics  and  the  planned 
curriculum  outline. 

Prior  to  completion  of  the  design  of  the  Simu¬ 
lation/Game  Subsystem,  the  model  requires  develop¬ 
ment  of  various  models  of  reality.  This  develop¬ 
ment  requires  a  definition  of  reality  in  the  form 
of  a  conceptual  model  (Activity  3.6),  the  develop¬ 
ment  and  documentation  of  an  operational  model 
(Activity  3.7)  and  the  evaluation  of  the  model,  in 
terms  of  completeness,  content  validity,  level  of 
specificity,  and  fidelity  (Activity  3.8).  Once 
again,  throughout  the  activities  culminating  in  the 
Simulation/Game  Subsystem  design,  there  should  be 
constant  reference  to  the  goal  of  the  system  as 
evidenced  by  the  learning  objectives  and  other  prod¬ 
ucts  of  earlier  activities.  In  addition,  it  is 
essential  that  these  activities  result  in  detailed 


documentation  for  the  model,  including  a  bibliogra¬ 
phy  and  list  of  references  and  a  record  of  the 
decision  processes  involved  in  model  development 
(i.e.,  assumptions  and  rationale). 

The  Critique/Feedback  Subsystem  is  primarily 
concerned  with  student  and  game  performance  moni¬ 
toring  and  evaluation.  The  design  of  the  Critique/ 
Feedback  Subsystem  should  provide  preliminary  an¬ 
swers  to  such  questions  as:  How  will  the  instructor/ 
controller  use  the  game  and  instructional  materials 
to  best  instructional  advantage?  What  should  be 
the  methods  of  recording  critical  events,  decisions 
and  outcomes? 

The  Implementation  Subsystem  design  should 
provide  for  complete  implementation  guidance  for 
instructors,  controllers  and  players.  The  design, 
therefore,  should  include  development  of  a  list  of 
tasks  selected  for  training  and  a  set  of  learning 
objectives  for  those  positions. 

Step  4.0  -  Develop  Initial  Prototype 

This  step  is  illustrated  in  Figure  4.  The  sub¬ 
system  designs  from  3.0  provide  the  primary  inputs 
to  the  development  activities  in  Step  4.0. 
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STEP  4.0  -  DEVELOP  THE  INITIAL  PROTOTYPE 


Figure  4.  Step  4.0  -  Develop  the  Initial  Prototype 


The  Simulation/Game  Subsystem  development 
should  result  in  development  of  the  game  rules  and 
procedures  and  physical  components.  Model  hardware 
and  software,  as  required,  will  also  be  developed 
during  this  activity.  Before  completion  of  develop¬ 
ment  of  the  Instructional  Subsystem,  Activity  4.3, 
the  Simulation/Game  Subsystem  should  be  evaluated 
in  relation  to  the  design  criteria  of  workability, 
playability,  validity  and  fidelity.  The  Simulation/ 
Game  Subsystem  evaluation  report  will  provide  valu¬ 
able  input  to  the  development  of  the  Instructional 
Subsystem;  particularly  in  the  development  of  sys¬ 
tem  training  capabilities  and  limitations. 

In  addition  to  the  lesson  plans,  scenarios,  and 
training  capabilities  and  limitations,  the  Instruc¬ 
tional  Subsystem  should  provide  a  matrix,  or  cross¬ 
index,,  of  learning  objectives  and  lesson  plans. 

The  development  of  the  Critique/Feedback  Sub¬ 
system  in  Activity  4.4  will  result  in  the  instruc¬ 
tions  and  procedures  necessary  to  evaluate  student 
performance.  Because  the  instructors  and  control¬ 
lers,  as  well  as  players,  will  be  principal  users 
of  the  Critique/Feedback  Subsystems,  the  develop¬ 
ment  of  this  subsystem  can  be  expected  to  impact 
the  development  of  learning  objectives  for  these 
positions  and,  therefore,  the  design  and  develop¬ 
ment  of  the  Implementation  Subsystem. 


The  Implementation  Subsystem  developed  through 
Activity  4.5  will  consist  of  instructor's  guides, 
controller's  guides,  player's  guides  and  the  exec¬ 
utive  guide  or  executive  summary. 

Upon  completion  of  initial  subsystems  develop¬ 
ment,  the  subsystems  must  be  integrated  into  an 
initial  prototype  training  system.  Integration 
activity  should  point  to  additional  changes  required 
in  the  initial  prototype  subsystems  and  components 
prior  to  commencement  of  initial  prototype  evalua¬ 
tion  in  Step  5.0. 


Step  5.0  -  Conduct  Initial  Evaluations 


The  primary  purpose  of  Step  5.0  is  the  overall 
effectiveness  of  the  initial  prototype  system  prior 
to  a  decision  to  produce  limited  quantities  of  the 
prototype.  These  evaluations  can  best  be  arranged 
in  three  phases:  (1)  in-house  evaluations; 

(2)  appraisal  by  subject  matter  experts;  and  (3)  one 
or  more  sets  of  external  evaluations. 


The  initial  evaluations  should  be  directed  to¬ 
ward  answering  questions  such  as: 


•  Are  the  lesson  materials  and  game  mutually 
supportive  of  the  prescribed  learning  objec¬ 
tive? 


•  Do  Che  Instructor,  controller  and  player 
guides  provide  sufficient  guidance  for  game 
play,  player  performance  evaluation  and 
critique? 

•  Is  the  game  reliable  in  that  the  simulation 
model  yields  consistent  results? 

•  Does  the  system  exhibit  the  degree  of  fidel¬ 
ity  intended  (l.e.,  are  the  simulations 
realistic  and  reflective  of  the  real  world)? 

•  Is  the  system  playable  in  a  physical  sense? 
Are  the  procedures  easily  understood  and 
does  the  system  meet  prescribed  criteria  for 
such  considerations  as  readability,  time  to 
play,  and  physical  characteristics?  Is  the 
game/simulation  interesting? 

•  Does  the  system  have  validity  in  terms  of 
reasonableness,  realism,  and  comprehensive¬ 
ness?  Is  the  appropriate  mix  of  simulation, 
automation,  activity  and  player  interaction 
provided? 

In  house  evaluation  should  be  an  extension  of 
the  iterative  processes  of  design,  develop,  test 
and  revise  begun  in  earlier  steps.  A  design  review 
checklist  should  be  developed  for  use  in  this  acti¬ 
vity,  which  can,  and  should,  begin  prior  to  comple¬ 
tion  of  the  initial  prototype  system. 

Subject-matter  expert  appraisal  can  be  accom¬ 
plished  with  the  assistance  of  the  Fleet  Project 
Team,  Reserve  Training  Units,  Tactical  Training 
Groups,  or  similar  activities,  as  appropriate.  It 
will  be  necessary  to  carefully  plan  and  schedule 
this  activity,  as  well  as  to  provide  a  complete 
package  of  data  collection  materials  and  appraisal 
guidance. 

The  third  activity  in  this  step  requires  one  or 
more  series  of  external  evaluations.  Actually, 
"external  evaluation"  is  a  misnomer  in  that  these 
activities  involve  implementation  of  the  system 
usinc  representative  members  of  the  target  popula¬ 
tion.  An  intensive  data  collection  effort  is  re¬ 
quired,  focusing  on  each  aspect  of  the  training 
sy-*  m.  Depending  upon  the  results  of  the  first 
x.  nl  evaluation,  a  second  or  even  third  evalu- 
'  may  be  required. 

'p  6.0  involves  production  of  limited  numbers 
of  t.  rototype  system.  The  initial  activity  is 
the  development  of  a  production  strategy,  to  include 
identification  of  government  and  contractor- 
furnished  "■■'terial  requirements  and  a  production 
plan.  Su  equent  activities  require  the  prepara¬ 
tion  of  p.ocurement  packages  and  tasking  for  cogni¬ 
zant  government  activities.  The  remaining  activi¬ 
ties  ii.  s  step  involve  contract  management  and 
completion  of  the  prototype.  Step  6.0  is  an  adap¬ 
tation  of  the  "fly  before  buy  concept"  employed 
with  success  in  other  areas. 

The  prototype  training  system  is  implemented 
in  Step  7.0.  The  first  activity  is  to  develop  an 
implementation  plan  that  will  establish  the  sched¬ 
ule  and  procedures,  and  assign  responsibilities  for 
prototype  implementation.  Prior  to  distribution  of 
the  prototype  systems,  one  or  more  courses  of  in¬ 
struction  will  be  required  for  those  personnel  as¬ 
signed  as  Instructors/controllers. 


A  second  evaluation  step  (step  8.0)  should 
follow  distribution  of  the  prototype  games  to  se¬ 
lected  units.  This  evaluation  should  be  conducted 
by  unbiased  sample  groups  from  the  target  popula¬ 
tion.  The  primary  purpose  here  is  to  evaluate  the 
training  system  in  its  intended  setting  and  without 
interferences  on  the  part  of  the  system  developers. 
Final  modifications  to  the  training  system  may  be 
required,  based  on  analysis  of  the  utilization 
data. 

Steps  9.0  and  10.0  of  the  development  model 
involve  production  and  implementation  of  the  oper¬ 
ational  training  system.  Step  11.0  provides  for 
"final"  evaluation  over  the  long-term  following 
full-scale  implementation.  This  evaluation  can, 
and  should  be  conducted  using  a  variety  of  infor¬ 
mation  sources,  including  regular,  feedback,  period¬ 
ic  sampling  of  users,  questionnaires  and  personal 
observations.  A  specific  goal  of  the  ongoing  eval¬ 
uation  activities  should  be  to  ensure  that  the  sys¬ 
tem  game,  scenarios  and  courseware  continue  to 
accurately  reflect  real-woild  operational  capabili¬ 
ties  and  doctrine. 

Step  12.0  involves  the  various  well-established 
activities  associated  with  providing  continuing 
management  support/services  to  the  users  of  the 
training  system. 

DEVELOPMENT  OF  DATA  PROCESSING  REQUIREMENTS 

As  mentioned  earlier,  the  design  of  a  wargame- 
based  training  system  may  require  use  of  a  digital 
computer.  Use  of  the  computer  is  generally  speci¬ 
fied  for  many  reasons:  to  speed  up  play:  to  allow 
for  utilization  of  more  sophisticated  models;  to 
permit  storage  and  use  of  much  more  data;  to  pro¬ 
vide  for  more  complex  interaction  with  the  player; 
to  allow  for  easier  employment  of  stochastic  mod¬ 
els;  for  ease  of  gathering  statistical  and  analyti¬ 
cal  data;  and/or  to  eliminate  the  need  for  many 
human  player/decision-makers. 

Automated  wargames  fall  into  three  general 
categories  of  increasing  difficulty  and  implementa¬ 
tions!  sophistication:  Computer  Assisted,  Inter¬ 
active  and  Computerized  wargames.  In  each  of  these 
categories,  the  data  processing  (Dp)  requirements 
that  must  be  addressed  are:  i  processing,  fast  mem¬ 
ory,  input,  output,  software' language,  and  oper¬ 
ating  system.  This  section  will  address  a  method 
for  developing  an  accurate  needs  estimate  for  a 
generic  wargame-based  training  system's  data  pro¬ 
cessing  requirements.  It  is  assumed  that  once  an 
accurate  data  processing  needs  estimate  has  been 
developed,  implementation  of  these  requirements 
will  be  accomplished  in  the  generally  accepted 
fashion  for  computerized  training  devices  (or 
training  devices  with  embedded  computers),  and  ac¬ 
cording  to  the  established  procedures  for  such  de¬ 
vices,  Military  Standard  for  Trainer  System  Soft¬ 
ware  Development,  MIL-STD-1644  (TD)  7  March  1979. 

Proper  development  of  the  device  data  process¬ 
ing  requirements  requires  integrated  analysis  per¬ 
formed  in  a  linear,  staged  fashion  of  increasing 
effort  and  sophistication.  The  authors  have  devel¬ 
oped  a  nine  step  model  defining  the  data  processing 
requirements  effort  (See  Figure  5).  This  process 
will  answer  the  critical  questions  in  the  seven 
data  processing  areas  detailed  above. 


314 


DEVELOPMENT  OF  DATA  PROCESSING  REQUIREMENTS 


Figure  5.  Development  of  Data  Processing  Requirements 


In  the  processor  area,  analysis  and  tests  are 
conducted  which  integrate  size  and  weight  limita¬ 
tions  with  model  complexities  to  determine  the  re¬ 
quired  speed  of  computation.  These  tests  must 
necessarily  also  address  the  issue  of  language, 
operating  system,  and  mass  and  fast  memory.  These 
processes  will  define,  for  example,  whether  a 
micro-,  mini-  or  main-frame  computer  is  required. 
The  issues  of  higher  order  language  vs.  machine 
language  and  standard  off-the-shelf  vs.  a  tailored 
operating  system  will  also  be  addressed.  The  war- 
game  model  complexity,  supporting  data  requirements, 
number  and  sophistication  of  participants  and  en¬ 
vironment  will  be  analyzed  to  provide  inputs  to  the 
mass  and  fast  memory  requirements.  Options  avail¬ 
able  for  player/participant  inputs  include  type¬ 
writer  keyboard,  joysticks,  light  pens,  trackballs, 
resistance  pot.,  push  buttons/switches,  graphics, 
tablets  and  voice  recognition.  The  capabilities 
must  be  matched  against  the  wargame  input  require¬ 
ments  and  user  skills.  Typical  output  character¬ 
istics  include  printer/hardcopy  and  cathode  ray 
tube  units;  their  speed  of  operation,  number  of 
lines  and  characters  per  line,  graphics  resolutions 
(line  drawing  or  raster),  colors  and  the  number  of 
words/phrases  for  voice  output.  Output  require¬ 
ments  address  the  system's  need  for  communication 
of  instructions,  computation  and  interaction  re¬ 
sults,  and  situation  reports/plots  to  the  players 
and  controllers.  Data  access  speeds  and  transfer 
rates  within  the  processor  and  to  its  various  per¬ 
ipheral  devices  are  important  concerns  for  proces¬ 
sor  timing,  mass  and  fast  memory  selection,  and 
input/output  operations. 


Referring  again  to  Figure  5,  the  first  step  in 
the  development  of  the  wargame-based  training  sys¬ 
tem  data  processing  requirements  is  integration 
and  analysis  of  the  device  operational  requirements. 
This  activity  will  result  in  a  list  of  automation 
tasks  and  an  initial  DP  needs  estimate.  For  NAVTAG, 
this  process  resulted  in  preliminary  selection  of  a 
micro-computer  using  a  high  order  language  in  an  off- 
the-shelf  operating  system  environment. 

In  Step  2,  the  list  of  automation  tasks  is  re¬ 
viewed  in  the  context  of  the  initial  DP  needs  esti¬ 
mate  to  identify  the  high  risk  areas  for  the  chosen 
approach.  The  list  of  high  risk  areas  for  the 
NAVTAG  device  included  the  capability  of  a  micro¬ 
computer  to  store,  position  update  and  legibly 
geographically  plot  the  positions  of  an  adequate 
number  of  platforms  within  reasonable  time  and 
memory  limitations,  and  the  capability  of  micro¬ 
computer  peripheral  storage  devices  to  store  all  of 
the  required  information. 

The  third  step  of  the  development  effort  de¬ 
signs  tests  which  will  aid  in  resolution  of  the 
high-risk  issues.  A  representative  micro-computer 
and  disk  storage  system  was  selected  to  implement 
test  programs  for  NAVTAG.  Software  design  specifi¬ 
cations  were  developed  which  addressed  the  storage, 
position  update  and  geographical  plotting  of  a  vari¬ 
able  number  of  platforms.  In  general,  this  step 
results  in  test  specifications  designed  to  generate 
the  timing,  storage  and  graphics  data  necessary  to 
adequately  lower  the  initial  risk  level  for  the 
Identified  tasks. 
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In  the  fourth  step,  the  teat  specifications 
from  the  previous  step  are  implemented  in  software 
and  performed  on  the  representative  hardware.  The 
timing,  storage  and  graphics  data  and  results  gen¬ 
erated  by  the  test  are  captured  and  documented. 
These  data  are  analyzed  and  used  to  refine  the  ini¬ 
tial  estimates  of  the  DP  requirements  in  the  fifth 
step.  The  outputs  from  this  stage  are  the  interim 
DP  needs  estimate  and  the  documentation  of  the  re¬ 
solution  of  the  high-risk  issues.  It  is  at  this 
stage  that  high-risk  automation  tasks  can  be 
dropped,  modified  or  used  to  justify  more  capable 
equipment/software,  depending  upon  their  priority 
and  complexity,  Analysis  of  the  results  of  the 
NAVTAG  high-risk  area  software  demonstrated  that 
reasonable,  effective  and  low-cost  solutions  exis¬ 
ted  for  these  issues. 

With  the  high-risk  automation  issues  resolved, 
the  sixth  step  addresses  the  Identification  of  the 
major  automation  tasks  that  require  Integrated 
feasibility  testing.  In  this  step,  the  interrela¬ 
tionship  and  dependencies  of  the  major  tasks  and 
the  special  input/output  requirements  are  analyzed. 
Tills  study  produces  a  list  of  those  critical  links 
and  input/output  functions  that  require  feasibility 
testing  to  more  completely  define  the  system  DP  re¬ 
quirements.  For  the  NAVTAG  training  system,  major 
automation  tasks  were  movement,  electronics,  weap¬ 
ons  utilization  and  damage  assessment.  The  feasi¬ 
bility  issues  concerned  the  interrelationship  of 
these  varied  tasks  through  the  rules  of  the  manual 
game  and  the  common  database  required.  Of  concern 
was  the  computer  reaction  and  computation  times  for 
the  varied  tasks,  and  the  form  and  format  of  commu¬ 
nications  with  the  system  user(s). 

The  design  of  a  limited,  yet  integrated  feasi¬ 
bility  automation  and  test  plan  is  accomplished  in 
the  seventh  step.  The  tests  are  constructed  to 
validate  the  interim  DP  requirements  and  to  evalu¬ 
ate  the  feasibility  of  the  critical  linkage  and 
communication  concepts  contained  in  the  system  de¬ 
sign.  The  automation  tests  and  design  for  the 
NAVTAG  feasibility  model  were  developed  in  an  iter¬ 
ative  fashion  between  computer  scientists,  train¬ 
ing  and  education  specialists  and  a  naval  subject 
matter  expert.  This  approach  assured  validity  of 
design  prior  to  feasibility  model  implementation. 

It  was  decided  to  model  two  ships  for  surface  in¬ 
teractions  only  because  of  time  and  cost  consider¬ 
ations.  The  game,  however,  still  contained  all  of 
the  major  automation  tasks  to  be  tested. 

The  eighth  step  is  the  implementation  of  the 
feasibility  model  designed  in  the  previous  step, 
and  utilization  of  that  model  to  accomplish  the 
limited  testing  specified.  The  output  of  this  step 
is  the  feasibility  model  and  documentation  of  the 
success  of  the  tests,  including  data  on  the  perfor¬ 
mance  and  acceptability  of  the  critical  automation 
tasks,  linkages  and  communication  strategies.  It 
is  interesting  to  note  that,  in  spite  of  consider¬ 
able  subject  matter  expert  input  in  development  of 
the  NAVTAG  feasibility  model,  significant  rework  was 
required  after  the  initial  Implementation.  Rede¬ 
sign  was  necessary  to  achieve  correct  terminology 
usage;  to  assure  straightforward  and  complete  pre¬ 
sentation  of  the  results  of  computer  activities, 
and  background  data  and  options  available  to  the 
user.  The  requirement  for  this  redesign  pointed  to 
the  need  for  a  model  for  development  of  wargame- 
based  training  systems. 

The  ninth  and  final  step  includes  analysis  of 


the  feasibility  model  test  results  and  documenta¬ 
tion.  This  analysis  refines  and  completes  the  es¬ 
timate  of  the  vargame-based  training  system's  DP 
requirements.  This  step  also  results  in  the  Iden¬ 
tification  of  interrelationships  and  system  overall 
integration  and  communication  (input/output)  re¬ 
quirements.  The  analysis  of  the  NAVTAG  feasibility 
model  produced  many  significant  results  that  drama¬ 
tically  aided  in  procurement  of  the  pre-production 
prototype.  It  was  at  this  stage  that  the  need  for 
use  of  a  compilable  higher  order  language,  instead 
of  an  interpreted  language;  and  the  need  for  an  80 
column  (rather  than  a  AO  column)  display  was  demon¬ 
strated.  Other  important  results  Included  an  ac¬ 
curate  data  base,  fast  memory  and  source  code  esti¬ 
mates.  This  information  proved  to  be  of  great  use 
during  the  competition  prior  to  award  of  the  con¬ 
tract. 

Utilization  of  this  procedure  to  define  the 
wargame-based  training  system  data  processing  re¬ 
quirements  produces  a  low-risk,  low  cost  system, 
while  maintaining  the  desired  level  of  performance. 
This  process  saves  money  and  lowers  risk  through 
studied  efforts  to  improve  the  device  through 
early  identification  of  high-risk  areas  and  refine¬ 
ment  of  DP  requirements;  based  on  hardware  evalua¬ 
tions  and  roftware  development  and  testing.  A 
further  benefit  is  that  the  amount  and  complexity 
of  the  device's  final  source  code  can  be  much  more 
accurately  estimated. 


CONCLUSIOl’S 

This  paper  has  presented  a  candidate  model  for 
the  development  of  wargame-based  training  systems. 
This  model  is  evolutionary;  its  application  to  real 
world  situations  has  been  limited.  It  is  under¬ 
going  revisions  as  it  is  currently  being  used  and 
evaluated  for  future  applications. 

The  model  was  developed  based  upon  the  broader 
principles  exemplified  by  the  instructional  system 
development  model  and  the  large  body  of  research 
already  completed  on  instructional  games  and  simu¬ 
lations.  The  experiences  of  both  the  Army  and  Navy 
in  developing  wargames  for  training  during  the  past 
ten  years  was  also  a  significant  factor. 

The  need  for  a  detailed  development  model  for 
wargame-based  training  systems  was  recognized  by 
the  authors  after  they  jointly  became  involved  with 
the  development  of  the  Naval  Tactical  Game  (NAVTAG). 
It  was  also  considered  to  be  of  benefit  to  the 
various  services  and  to  industry  because  of  the  in¬ 
creased  interest  in  the  use  of  wargames  for  train¬ 
ing  and  the  numerous  potential  applications  for 
such  systems  throughout  the  Department  of  Defense. 

The  need  for  regular  proficiency  training  for 
tactical  decision-making  has  long  been  recognized. 
Wargames  are  one  way  of  providing  combat  leaders 
with  experience  in  the  operational  Impact  of  their 
decisions.  When  properly  developed  and  implemented, 
wargames  can  provide  valuable  training  to  tactical 
commanders  in  all  the  services.  However,  it  is  im¬ 
portant  that  the  reader  understand  the  authors  do 
not  offer  wargame-based  training  systems  as  a  pan¬ 
acea  for  all  tactical  training  problems.  Applica¬ 
tions  of  this  model  (or  appropriate  variations)  will 
increase  the  probability  that  such  instruction  will 
be  both  cost  and  training  effective/efficient.  The 
use  of  the  instructional  systems  development  model 
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as  a  basis  for  this  model  allows  for  simultaneous 
optimization  of  both  the  wargame  and  the  instruc¬ 
tional  system. 

Variations  of  this  model  are  being  used  as  the 
basis  for  development  of  a  number  of  wargame-based 
training  systems  at  the  Naval  Training  Equipment 
Center.  The  success  achieved  through  application 
of  the  model  to  date  has  been  excellent,  with  the 
Naval  Tactical  Came  (NAVTAC)  most  notable.  Spe¬ 
cific  examples  of  the  application  of  this  model  to 
NAVTAG  development  program  have  been  included  in 
this  paper,  where  appropriate,  to  further  explain 
and  demonstrate  the  intent  of  this  model.  Notable 
benefits  to  that  program  from  use  of  this  model 
have  been  reported.  A  subsequent  effort  will  ad¬ 
dress  the  application  of  a  variation  of  this  model. 
Including  each  of  the  twelve  steps  to  the  develop¬ 
ment  of  a  manual  wargame-based  training  system  for 
the  USMC. 

Consideration  should  also  be  given  to  utiliz¬ 
ing  the  model  presented  in  this  paper  for  defining 
the  data  processing  requirements  whenever  computers 
are  used  as  part  of  the  training  system. 

As  noted  earlier,  this  paper  does  not  attempt 
to  define  all  the  advantages  and  disadvantages  for 
using  wargame-based  training  systems,  nor  to  de¬ 
scribe  in  any  great  detail  the  media  selection  pro¬ 
cess  which  would  result  in  identification  of  the 
need  for  a  wargame-based  system.  It  is  the  authors’ 
opinion  that  the  criticality  of  the  duties  of  our 
tactical  decision-makers,  in  all  services,  requires 
that  any  and  all  reasonably  effective  approaches  be 
pursued,  and  that  no  one  solution  be  offered  as  a 
panacea  to  the  problems  of  training  those  combat 
leaders.  The  fact  that  wargames  -  or  wargame-based 
training  systems  -  can  be  used  to  provide  a  partial 
solution  to  those  needs  is  well  recognized.  The 
model  presented  herein  provides  one  possible  frame¬ 
work  for  achieving  those  goals  in  a  cost/training 
effective  and  efficient  manner.  The  validity  of 
this  model  will  be  tested  in  the  cauldron  of  real 
world  application  during  the  coming  year,  and  the 
results  reported  in  a  subsequent  paper. 
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BASED  ON  DIGITAL  STIMULATION 
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ABSTRACT 

Prior  techniques  in  EW  traininq  equipment  desiqns  have  used  either  the  approach  of  signal  injec¬ 
tion  Into  operational  EW  equipment  or  the  approach  of  software  real-time  modeling  of  the  EW  equipment 
and  environment.  The  analoq  injection  approach  preserves  the  siqnal  processing  characteristics  and 
anomalies  of  the  EW  equipment,  but  is  costly  and  difficult  to  maintain  and  keep  calibrated.  The  soft¬ 
ware  model  approach  is  easily  maintained  and  offers  excellent  simulation  repeatability,  but  the  real¬ 
ism  Is  limited  to  the  deqree  in  which  the  math  model  simulates  the  equipment  and  environment.  The  new 
approach  presented  in  this  paper  offers  the  realism  advantages  of  the  siqnal  injection  approach  plus 
the  repeatability  and  maintainability  of  the  software  modelinq  approach.  Instead  of  injectinq  analog 
RF  signals  or  video  level  signals  into  the  operational  EW  equipment  analoq  to  diqital  (A/D)  siqnal 
converter,  the  new  approach  bypasses  the  A/D  converter  and  injects  digitally  formatted  signals 
directly  into  the  EW  processor  data  collection  buffer.  This  approach  uses  qeneral  purpose  diqital 
computer  equipment  to  generate  the  real-time  diqital  pulse  data  and  uses  the  operational  EW  equipment 
to  process  the  data.  All  the  processing  characteristics  and  anomalies  are  preserved  in  the  EW  equip¬ 
ment,  and  the  repeatability  and  maintainability  of  diqital  versus  analog  siqnal  generators  are  pro¬ 
vided. 

INTRODUCTION 


This  paper  describes  a  new  and  unique 
trainer  design  concept  for  a  simulator  to 
be  used  in  traininq  pilots  and  electronic 
warfare  (EW)  officers  in  the  use  of  radar 
warning  system  (RWS).  Past  approaches 
have  either  employed  RF  level  stimulation, 
video  level  stimulation,  or  software  simu¬ 
lation  of  the  RWS.  However,  with  the 
state-of-the-art  trend  toward  digital  RWS 
and  digital  signal  processing,  a  new  ave¬ 
nue  has  been  opened  up,  digital  stimula¬ 
tion.  As  used  in  the  text  of  this  paper, 
digital  stimulation  is  defined  as  the  in¬ 
jection  of  digitized  radar  data  into  the 
direct  memory  access  (DMA)  buffers  of  an 
operational  RWS.  This  new  design  incor¬ 
porates  the  operational  siqnal  processor 
with  its  complex  software,  threat  library, 
I/O  units,  displays  and  control  boxes. 

The  operational  equipment  analoq  to  diqi¬ 
tal  signal  converter  is  replaced  by  an  in¬ 
terface  to  a  general  purpose  simulation 
computer.  The  simulation  computer  gener¬ 
ates  digital  words  which  the  RWS  signal 
converter  would  have  generated  under  the 
same  operating  conditions.  All  RWS  re¬ 
sponse  times,  reactions  to  ambiguous 
threats,  and  simulation  of  most  system 
anomalies  are  guaranteed,  since  the  actual 
on-board  signal  orocessinq  equipment  is 
used. 

Simulation  of  the  EW  environment  and 
accurate  modelinq  of  the  RWS  antennas,  re¬ 
ceivers,  and  siqnal  digitizing  modules  are 
performed  in  software  on  the  general -pur¬ 
pose  computer.  Real-time  line-of-siaht, 
relative  position,  aircraft  orientation, 
and  emitter  propagation  effects  are  also 
performed  by  this  simulation  computer  to 
provide  a  continuous  stream  of  information 
to  the  RWS  computer  and  signal  processing 
software. 

As  in  most  training  devices,  monitor¬ 
ing  and  control  of  the  simulated  threat 


environment,  RyjS  and  traininq  exercises 
are  provided  by  an  instructor's  console 
interfacing  with  the  simulation  computer. 

Whether  designed  for  application  as  a 
task  training  device  or  integrated  into 
operational  flight  or  weapon  system 
trainers,  this  digital  stimulation  concept 
should  provide  the  foundation  for  low 
cost,  realistic,  reliable  and  easily  main¬ 
tainable  EW  traininq  devices. 


REQUIREMENTS 


Performance  Requirements 


A  radar  warninq  system  trainer  (RWST) 
must  meet  a  number  of  performance 
requirements  to  be  an  effective  training 
device.  It  must  be  able  to  reproduce  (on 
a  real-time  basis)  all  the  visual  and  au¬ 
dio  cues  that  the  actual  system  produces 
in  the  same  conditions  of  flight  and 
threat  environments.  All  character  move¬ 
ments  or  strobes  on  displays,  lighted  leg¬ 
ends,  audio  tones  and  beeps,  and  system 
reaction  to  control  switches  must  function 
as  if  the  operational  equipment  were  ac¬ 
tively  being  used.  Realism,  accuracy,  and 
repeatability  of  the  cues  are  important  to 
the  trainee.  However,  the  deqree  of  real¬ 
ism  should  not  extend  beyond  the  threshold 
of  cue  discernment  for  most  pilots  for  the 
system  to  be  properly  cost-effective. 


The  specific  performance  requirements 
of  an  RWST  can  be  divided  into  several 
categories,  as  follows: 


•  Airborne  Equipment  Simulation 
All  displays,  controls  and  audio  tones  as- 
sociated  with  the  RWS  during  both  normal 
and  failure  modes  of  operation  must  be 
realistically  simulated,  includinq  siqnal 
processing  anomalies  such  as  delay  times, 
false  alarminq,  misidentif ications  and  DF 
wander . 
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•  EM  Environment  Simulation 
Radar  threat  models  need  to  be  simulated 
with  those  characteristics  which  Influence 
the  resocnse  of  the  RMS,  such  as  pulse  re¬ 
petition  Intervals,  transmlttlnq  frequen¬ 
cy,  scan  modulation,  correlation  of  mis¬ 
sile  quidance  uplinks,  and  Dulse  synchron¬ 
ization  with  other  threats. 


(F.CM)  must  be  taken  into  account,  alonq 
with  the  measurement  of  pilot  reaction 
times  to  onqaae  the  proper  E CM  or  make 
proper  evasive  aircraft  maneuvers  to  mini¬ 
mize  the  probability  of  kill. 

The  development  of  on-board  equipment 
simulation  requires  an  understandlnq  of 
the  elements  of  an  airborne  radar  sys- 


•  RMS  and  Environment  Interaction 
Emitter  mode  switchlnq  as  a  functTo'n  of 
ranqe  from  the  aircraft,  and  the  reaction 
of  emitters  to  electronic  countermeasures 


tern.  Fiqure  1  shows  the  physical  compo¬ 
nents  of  the  AN/ALR-67,  a  typical  airborne 
system  built  by  ATI.  Fiqure  2  presents  a 
hlqhly  simplified  block  dlaqram  of  a 
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representative  airborne  RWS  consistinq  of 
directional  antennas,  RF  cables,  radar  re¬ 
ceivers,  a  siqnal  diaitizer,  a  siqnal 
analvzer,  a  threat  library,  a  control/in¬ 
dicator  panel,  an  azimuth  display,  and 
audio  output .  An  FCM  system  may  be  a  part 
of,  or  directlv  interfaced  with,  the  RWS, 

Free-snace  radar  siqnal s  are  picked 
up  by  the  antennas,  sent  throuqh  RF  cables 
to  receivers,  and  converted  by  video  am¬ 
plifiers  into  voltaqe  level  siqnals.  The 
siqnale  are  then  diqitized  into  a  set  of 
data  words  containinq  time  of  arrival,  an- 
qle  of  arrival,  amplitude,  pulse  width, 
transmittinq  frequency  or  hand,  and  var¬ 
ious  taqs  or  flaqs.  The  diqital  pulses 
are  deinterleaved  and  processed  bv  the 
analyzer  to  determine  nulse  repetition  in¬ 
tervals  (PR1),  amplitude  modulation  char¬ 
acteristics,  and  other  siqnal  siqoa- 
tures.  These  processed  siqnals  are  then 
compared  to  the  siqnatures  of  known 
threats  in  order  to  make  an  identification 
and  assion  the  appropriate  display  sym¬ 
bol.  The  azimuth  display  and  control/in¬ 
dicator  panel  are  driven  by  the  analyzer 
to  display  to  the  pilot  the  presence  of 
the  detected  threat,  siqnal  anqle  of  ar¬ 
rival,  relative  threat  lethality,  and  sys¬ 
tem  status.  Fiqure  3  shows  a  typical  azi¬ 
muth  display  from  a  modern  RWS  system.  In 
so^e  systems,  audio  identification,  alert, 
and  warninq  tones  are  also  produced. 

Whether  the  elements  of  such  a  system 


are  simulated  or  stimulated,  certain  char¬ 
acteristics  must  be  preserved  if  the  pilot 
is  to  he  given  realistic  cues.  As  a  mini¬ 
mum,  the  followinq  factors  must  be  taken 
into  account  to  achieve  realistic  system 
performance: 

•  Antennas  -  Three-dimensional  re¬ 
ception  qain  pattern  as  a  function  of  ele¬ 
vation  and  azimuth  incident  anqles  and 
siqnal  carrier  frequency;  movement  dyna¬ 
mics:  and  maskinq  of  the  aircraft 

•  RF  Cab  1 es  -  Signal  power  loss  as 
a  function  of  cable  length  and  siqnal  fre¬ 
quency 

•  Receivers  -  Milliwatt  inout  to 
millivolt  output  transfer  characteristics 
for  video  amplifier:  triplexer  or  quadru- 
plexer  filter  skirts:  characteristics  of 
local  oscillators  or  mixers 

•  Signal  Digitizer  -  Millivolt  in¬ 
put  to  diqital  word  output  transfer  func¬ 
tions,  including  effects  of  siqnal  walk¬ 
throughs  and  dead  times  between  pulses 

•  Analyzer  -  Siqnal  processing  al¬ 
gorithms,  "time  delays,  reaction  to  missinq 
pulses  and  pulse  walkthrough  effects: 
threat  ambiquit.y  resolutions;  reaction  to 
pilot  mode  selection 

•  Threat  Library  -  Specific  threat 
parameter  TTmTTs 
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figure  3.  Typical  RWS  Azimuth  hisolav 


•  Displays  and  Controls  -  Refresh 
rates,  update  rates,  s.vmboloqy,  intensity 
effects,  synchronization  of  display  sym¬ 
bols.  with  lighted  legends  and  audio  tones 

The  requirements  for  equipment  simu¬ 
lation  also  apply  to  threat  environment 
simulation.  All  possible  modes  of  a 
threat  emitter  that  may  emanate  from  a  pi¬ 
lot's  warning  equipment  must  be  simulated 
to  provide  a  sufficient  level  of  real¬ 
ism.  Some  of  the  threat  environment  char¬ 
acteristics  that  must  be  considered  are: 

•  Radiating  Beam  Patterns  -  Beam 
shape,  and  radiated  power  as  function  of 
modes  of  operation 

•  Scan  Pattern  -  Searching  and 
tracking  scan  patterns  such  as  circular, 
conical  and  raster  scans 

•  Pulse  Modulation  -  Hiqh  and  low 
pulse  repetition  frequency  modes;  stag¬ 
gered,  jittered  and  pulse  frequency  modu¬ 
lation 


•  Frequency  Modulation  -  Band 
switching  and  frequency  hoppinq  effects 

•  Guidance  Signals  -  Realistic 
pattern  and  correlations 

Simulation  of  the  RWS/environment  in¬ 
teraction  may  vary  widely.  A  simple  model 
would  require  that  each  threat  have  pro¬ 
grammable  modes  of  operation  as  a  function 
of  range  to  the  aircraft,  includinq  line- 
of-sight  and  space  loss  propagation  ef¬ 
fects.  A  more  sophisticated  model  would 
program  threat  emitters  to  change  modes  of 
operation  and  interact  witti  other  emitters 
as  a  function  of  passive  and  active  ECM 
employed  by  the  pilot  of  the  simulated 
aircraft.  It  should  also  be  noted  that 
whatever  the  degree  of  intet active  simula¬ 
tion  required  for  an  RWST,  the  effects  may 
need  to  be  coordinated  with  other  training 
or  simulation  systems.  For  example,  the 
number  of  threats,  location,  orientation 
and  modes  of  operation  must  be  coordinated 
with  simulated  visual  and  radar  land  mass 
systems,  when  such  systems  are  part  of  a 
trainer. 
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Instruction  Control  Requirements 

To  be  an  effective  training  tool,  an 
RWST  must  possess  certain  on-line  and  off¬ 
line  Instructor  control  features.  The  re¬ 
quired  on-line  features  Include: 

•  A  tactical  situation  display 
(TSO)  showing  the  current  position  of  all 
threats  in  the  environment  along  with  the 
RWS  aircraft  position  and  heading 

•  Means  of  monitoring  and  con¬ 
trolling  the  location  and  modes  of 
operation  of  each  simulated  threat 

•  Real-time  duplication  of  dis¬ 
plays  and  audio  tones  sensed  by  the 
trainee. 

•  Means  of  injecting  and  deleting 
simulated  RWS  equipment  malfunctions 

•  Mission  record  and  playback  con¬ 
trol  functions,  along  with  situational 
freeze  and  reset  controls 

A  part  from  the  on-line  instructor 
control  features  which  provide  the  general 
training  exercise  and  simulated  mission 
monitoring  and  control  functions,  there  is 
the  category  of  off-line  instructor  con¬ 
trol  features.  The  creation  of  training 
scenarios  is  the  primary  off-line  func¬ 
tion.  Prior  to  training  exercises,  the 
instructor  must  be  able  to  conveniently 
build,  modify,  or  select  simulated  threat 
environments.  This  requires  an  interac¬ 
tive  threat  data  base  editing  capability 
as  well  as  mass  storage  facilities  to  hold 
and  reload  previously  constructed  scenar¬ 
ios. 

Interface  Requirements 

The  RWST  may  need  to  interface  with  a 
number  of  the  following  external  systems: 

•  Host  simulator  instructor's 
console 

•  Auxiliary  control  terminals 

•  Host  simulator  computer  system 

•  Earth  effects  simulator 

•  ECM  subsystems 

•  Other  detection  subsystems 

•  Weapons  subsystems 

•  Special  test  facilities 

In  addition  to  the  instructor's  con¬ 
sole,  which  is  generally  an  inherent  part 
of  the  RWST  (as  was  described  in  the  pre¬ 
vious  section),  there  may  be  an  integrated 
tactical  instructor's  control  station  as¬ 
sociated  with  the  host  flight  or  weapon 
system  trainer.  In  such  applications,  the 
RWST  must  work  in  a  fully  integrated  mode 
with  other  systems,  sharing  a  common 


threat  environment  data  base  and  set  of 
instructor  control  functions.  A  trainee 
can  optimize  his  mission  effectiveness 
only  when  his  RWS  proficiency  is  mastered 
in  coordination  with  other  on-board  sys¬ 
tems  in  a  variety  of  tactical  situations. 

In  addition  to  the  regular  instructor 
consoles,  some  RWSTs  may  have  special  aux¬ 
iliary  control  terminals  to  permit  maneu¬ 
vering  and  control  of  special  emitters 
(e.g.,  airborne  interceptors)  in  real  time 
to  simulate  realistic  dynamic  engagement 
scenarios. 

Another  important  interface  is  with 
the  flight  simulation  computer.  RWST 
training  is  generally  performed  in  con¬ 
junction  with  flight  training  to  allow  the 
trairee  to  perform  simultaneous  'ehiole 
control  alonq  with  EW  engagement.  The 
host  simulator  computer  must  provide  per¬ 
iodic  information  to  the  RWST  indicating 
host  vehicle  position,  speed,  heading,  al¬ 
titude,  attitude  (e.g.,  pitch,  roll,  and 
yaw  of  aircraft)  and  other  pertinent  ve¬ 
hicle  effects.  In  turn,  the  RWST  will 
provide  to  the  host  vehicle  system  simula¬ 
tor  inputs  that  might  affect  subsystems 
within  the  host  vehicle  simulator  (e.g., 
acoustic  siqnals  to  the  operator,  sound 
effects,  status  of  threats  as  they  would 
affect  radars  in  the  vehicle). 

If  the  host  vehicle  simulator  has  a 
visual  display  to  provide  the  trainee 
realistic  images  simulating  the  opera¬ 
tional  environment,  it  is  extremely 
important  that  the  EW  engagement  stays 
synchronized  with  the  appropriate 
images.  For  example,  to  simulate  the 
destruction  of  emitter  sites,  it  is 
necessary  to  coordinate  the  visual  images 
with  loss  of  signal  on  the  RWS  display  and 
possibly  loss  of  target  on  the  host 
simulator's  radar  landmass  display. 

The  earth  effects  simulator  provides 
inputs  to  the  EW  simulator  on  terrain  or 
environmental  factors  which  will  establish 
whether  emitters  are  hidden  and  whether 
their  respective  signals  may  have  delays, 
attenuation,  distortion,  and/or  multiple 
reflections . 

If  ECM  effects  are  to  be  included, 
the  deployment  of  ECM  by  the  trainee  must 
be  sensed  and  the  likely  effects  of  emit¬ 
ter  track  loss  or  mode  changes  must  be 
superimposed  on  the  threat  scenario.  In 
addition,  the  effects  of  chaff  on  the  RWS 
must  also  be  imposed  onto  the  RWST  simu¬ 
lation. 

The  RWST  must  also  be  designed  to 
permit  special  test  subsystems  to  be 
hooked  up  to  provide  for  various  checkout 
and  maintenance  operations.  Such  test 
subsystems  might  be  used  to  verify  the 
operational  readiness  of  the  trainer,  and 
to  aid  in  isolation  and  repair  of  failed 
trainer  components. 
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Other  Requirements 


Because  such  trainers  are  expensive 
and  generally  not  built  in  larqe  quan¬ 
tities,  a  replacement  system  or  replace¬ 
ment  parts  cannot  easily  be  found.  These 
systems  must  be  designed  for  high  avail¬ 
ability.  This  calls  for  high  reliability 
and  ease  of  maintenance:  in  essence,  a 
long  MTBF  and/or  low  MTTR, 

By  its  nature,  the  design  of  the  RWS 
trainer  calls  for  a  system  configuration 
that  is  flexible,  readily  modifiable,  and 
highly  adaptable.  From  the  onset,  the  de¬ 
sign  should  be  geared  for  change.  This  is 
true  for  several  reasons: 

®  Many  of  the  functions  to  he  sim¬ 
ulated  cannot  be  accurately  defined  durinq 
the  design  phase  (e.g.,  RWS  detection 
characteristics  as  affected  by  aircraft 
installation  and  EOi  factors).  These 
functions  should  remain  variable  until 
sufficient  experience  is  gained  in  flying 
one  or  more  aircraft  in  a  tyoical  experi¬ 
mental  ranqe. 

•  The  threat  environment  and/or 
the  software  elements  of  the  operational 
RWS  system  are  likely  to  change  with  time, 
and  the  system  performance  will  need  to 
change  accordingly. 

•  The  EW  system  may  be  configured 
within  a  number  of  different  type  host  ve¬ 
hicles  or  interfaced  with  chanqinq  equio- 
ments  (e.g.,  different  .iammers  or  weapons) 
over  the  lifetime  of  the  system,  requiring 
the  desiqn  of  subsystem  interface  as  well 
as  the  operational  characteristics  of  the 
system  to  vary  with  time. 


The  trade-off  in  performance  qained 
for  added  cost  of  realism  is  a  major  re¬ 
quirement  in  the  practical  desiqn  approach 
to  RWSTs. 

PREVIOUS  DESIGN  APPROACHES 

A  number  of  design  appro  iches  have 
been  used  to  simulate  radar'  v.  rninq  equip¬ 
ment  and  threat  environments,  lunqing  from 
various  methods  of  signal  injection  into 
operational  RWS  hardware  to  total  software 
model inq  of  the  operational  hardware.  The 
former  approach  has  been  called  "stimula¬ 
tion"  and  the  latter,  "simulation."  Hav¬ 
ing  designed  equipment  using  both  of  these 
approaches.  Applied  Technology  is  inti¬ 
mately  familiar  with  the  corresponding  ad¬ 
vantages  and  disadvantages.  Applied 
Technology  has  developed  two  software 
RWSTs  currently  in  use  with  operational 
fliqht  trainers,  for  both  the  Canadian  Air 
Force  and  another  NATO  country. 

Signal  injection,  or  stimulation,  can 
be  at  either  the  radio  frequency  (RF)  or 
video  (as  converted  bv  the  receiver) 
levels.  In  the  RF  approach,  the  RF 
siqnals  are  injected  into  the  operational 
receivers  (see  fiqure  2).  The  advantage 
of  this  approach  is  that  most  of  the 
operational  equipment  is  used  without 
modification.  If  a  part  fails  in  the 
trainer,  it  is  possible  to  replace  it  with 
aircraft  parts.  This  approach  also 
provides  for  many  of  the  subtle  mixing  and 
walkthrough  requirements  of  siqnals  which, 
if  not  met,  could  seriously  affect  the 
displays  and  audio  cues  from  a  warning 
system. 


For  these  and  other  reasons,  it  is 
necessary  that  the  trainer  operation  be 
easily  adaptable  to  meet  broad  and  chanq¬ 
inq  requirements. 

This  adaptability  factor  brinqs  up  a 
special  additional  problem:  configuration 
management.  Provision  must  be  made  in  the 
desiqn/development/adaptation  cycle  for 
setting  up  and  maintaining  adequate  knowl¬ 
edge,  control  and  documentation  on  the 
configuration  of  each  version  of  the 
trainer  that  is  built  and  separately  in- 
stal led . 

Since  proper  traininq  requires  real¬ 
istic  threat  data,  the  EW  trainer  also  re¬ 
quires  special  security  measures  and  con¬ 
trolled  access  of  the  classified  data  to 
authorized  individuals. 

As  for  other  systems,  cost  is  a  major 
consideration  in  the  design.  In  general, 
increasing  the  realism  of  simulation  (rep¬ 
resenting  the  threat  environment,  propa¬ 
gation  and  reflection  effects,  effects  on 
the  signal  of  aircraft  motion  and  body 
shadowinq,  and  the  different  anomalies; 
all  of  which  may  occur  to  the  detection 
process  in  real  life)  calls  for  more  com¬ 
puter  power,  more  sophisticated  algor¬ 
ithms,  and  much  specialized  hardware. 


The  disadvantages  of  the  RF  level 
stimulation  approach  are  numerous.  It  is 
difficult  and  costly  to  obtain  a  good  dy¬ 
namic  power  ranqe,  pulse  width,  and  rise 
time  characteristics  and  be  able  to  mix 
several  independent  signal  sources.  Ex¬ 
tensive  calibration  procedures  are  re¬ 
quired  to  prevent  degraded  performance. 
Individual  RF  sources  are  costly  and  have 
a  ooor  MTBF.  A  central  computer  system  is 
needed  to  control  the  individual  emitter 
sources  and  compute  the  antenna  gain  pat¬ 
tern  effects  on  the  RF  injected  signals. 

The  video  signal  stimulation  approach 
models  the  receiver  and  antenna  transfer 
characteristics  and  squirts  video  level 
signals  into  an  analyzer  or  siqnal  digi¬ 
tizer  (see  figure  2).  The  advantage  of 
this  technique  is  that  it  also  uses  oper¬ 
ational  equipment.  Thus,  if  tl  < eat  li¬ 
brary  or  operational  analyzers  are  changed 
on  the  aircraft,  they  can  be  installed  in 
the  trainer  with  little  or  no  reprogram¬ 
ming  requirements.  Signal  mixing  and 
walkthrough  can  be  realistically  estab- 
1 ished. 
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The  disadvantages  of  this  approach 
are  similar  to  those  of  RF  level  stimula¬ 
tion,  but  are  not  so  extensive.  Banks  of 
pulse  and  scan  generators  must  be  mixed  in 
a  very  accurate  manner  to  ensure  proper 
walkthrough  characteristics.  The  video 
sources  must  be  calibrated  to  ensure  pro¬ 
per  power  range  and  angle-of-arrival  simu¬ 
lation.  However,  they  have  a  much  better 
MT8F  and  do  not  drift  out  of  calibration 
as  fast  as  the  RF  sources.  Like  the  RF 
stimulation  approach,  a  computer  system  is 
needed  to  proqram  and  control  the  video 
sources.  The  antenna  gain  patterns,  cable 
losses  and  receiver  transfer  characteris¬ 
tics  must  be  modeled  into  the  stimulated 
signals  to  provide  proper  realism. 

The  software  simulation  approach  has 
the  primary  advantage  of  a  low  recurrinq 
cost  for  multiple  training  devices,  plus  a 
very  good  MTBF.  Use  of  general-purpose 
computers  to  model  the  entire  RWS  in  soft¬ 
ware  (with  the  exception  of  the  displays 
and  control  panel  in  the  cockpits) 
provides  other  significant  advantages. 

The  ability  to  readily  program  and 
reprogram  any  desired  operational 
condition  and/or  anomaly  makes  this 
approach  very  flexible.  Reproducing  the 
same  indications  for  record/playback 
purposes  is  easily  achievable  in  digital 
computer-based  systems.  Simulating 
equipment  failures  or  common  malfunctions 
is  also  easy  in  the  software  simulation 
approach. 

The  main  disadvantages  of  this  ap¬ 
proach  lie  in  two  areas.  First,  there  is 
the  difficulty  in  assuring  that  the  soft¬ 
ware  model  can  realistically  reproduce  the 
time  responses,  false  alarms,  OF  wander, 
signal  modifications  and  other  anomalies 
associated  with  the  operational  equip¬ 
ment.  Second,  changinq  any  part  of  the 
operational  hardware  in  the  aircraft  would 
require  a  reprogramminq  effort  in  the 
RWST . 

SELECTED  APPROACH 

The  selected  aporoach  ,  which  is  cur¬ 
rently  under  development  at  Applied  Tech¬ 
nology,  is  outlined  in  figure  4  and  de¬ 
scribed  in  the  following  paragraphs.  The 
block  diaqram  represents  the  functional 
make-up  of  the  system  in  its  integrated 
mode  of  operation  (i.e.,  in  conjunction 
with  fliqht  or  weapons  system  traininq). 
The  stand-alone  mode  of  system  operation 
will  be  briefly  described  later. 

During  the  integrated  mode  of  opera¬ 
tion,  there  is  a  real-time  input  providing 
the  host  vehicle's  current  oosition  and 
attitude  as  it  is  being  flown  through  => 
simulated  gaminq  area.  The  position  and 
attitude  data  could  also  be  derived  from  a 
prerecorded  simulated  fliqht. 

The  "siqnal  selection  model"  using 
this  information  accesses  the  "emitter 
model"  to  determine  which  emitters  would 
be  candidates  at  this  point  in  space  for 
the  RWS  to  detect,  process,  and  present  on 


the  trainee's  display.  The  emitter  model 
is  essentially  a  large  data  base  which 
contains  Information  on  location,  mode  of 
operation,  and  operational  characteristics 
of  all  emitters  to  be  included  in  the 
traininq  scenario.  In  addition  to  earth- 
referenced  position  information,  the  emit¬ 
ter  model  indicates  when  the  emitter  is 
active  and  the  radar  mode  in  which  it  is 
operating.  The  scan  and  beam  patterns  of 
the  emitters  as  a  function  of  time,  plus 
transmission  characteristics  such  as  pulse 
repetition  intervals,  radiated  power  and 
frequency  are  also  identified.  If  the 
emitters  are  mobile,  as  are  aircraft  In¬ 
terceptors  (AI),  there  will  be  special 
control  consoles  for  maneuvering  them  dur¬ 
ing  simulation.  In  any  event,  the  in¬ 
structor  will  be  able  to  monitor  and  mod¬ 
ify  the  status  and  mode  of  any  emitter 
during  the  training  session. 

Using  factors  such  as  the  relative 
position  of  the  emitters  with  respect  to 
the  location  of  the  trainee's  vehicle,  and 
the  emitter's  current  status,  the  "siqnal 
selection  model"  determines  which  emitters 
are  visible  to  the  RWS,  their  effective 
received  power,  and  their  relative  angle 
with  respect  to  the  host  vehicle's 
course.  All  dynamic  emitter  selection  is 
made  available  to  the  instructor's  console 
in  the  form  of- a  tactical  situation  dis¬ 
play. 

The  "propagation  effects  model,"  for 
every  siqnal,  computes  what  effective  siq¬ 
nal  strength  would  be  detected  at  the  air¬ 
craft  antennas  based  on  space  losses  and 
the  respective  sending  and  receiving  an¬ 
tenna  lobe  characteristics.  Any  other  ef¬ 
fects,  such  as  moisture  absorption  or  sig¬ 
nal  reflections,  are  also  introduced  at 
this  point. 

The  "aircraft  dynamics  model"  modi¬ 
fies  the  received  signal  characteristics 
based  on  the  current  attitude  of  the  simu¬ 
lated  aircraft.  A  six-degree-of-freedom 
model  is  used  to  compute  the  relative  in¬ 
cident  angles  of  all  incominq  siqnals  with 
respect  to  the  receiving  antennas.  The 
output  of  this  model  goes  into  the  "an¬ 
tenna  receiver  model,"  and  can  be  sent  to 
other  subsystems. 

The  "antenna/receiver  model"  simu¬ 
lates  the  antenna  gain  patterns  as  a  func¬ 
tion  of  relative  incident  angles  received 
from  the  "aircraft  dynamics  model."  The 
emitter  transmitting  frequency  also  in¬ 
fluences  the  antenna  qains.  Aircraft 
masking  effects  are  included  in  the  an¬ 
tenna  model,  as  well.  The  transfer  func¬ 
tion  associated  with  the  RWS  receivers  are 
a  part  of  this  model,  and  include  such  ef¬ 
fects  as  crystal  video  pulse  compression, 
pulse  shadowing  effects  and  recovery 
times . 

The  "effects  generator"  includes  the 
transfer  functions  inherent  in  the  param¬ 
eter  digitizing  unite  of  the  radar  warninq 
system.  Various  effects  can  be  input  on  a 


. 
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statistical  and/or  operator -control  led 
basis  to  simulate  missing  signals, 
spurious  siqnals,  failures  or  other 
special  effects.  The  "effects  generator" 
also  interceots  control  siqnal  inputs  from 
the  trainee  (i.e.,  RWS  control  panel 
switches)  and  modifies  system  operation 
accordingly.  The  output  of  the  "effects 
generator"  is  real-time,  digitized  data 
words  representing  the  radar  signal  data 
which  the  RWS  signal  digitizer  would  send 
into  the  RWS  siqnal  processor.  The 
"effects  generator"  injects,  at  the  appro¬ 
priate  rate,  the  diqitized  emitter  para¬ 
meters  for  the  RWS  software  to  process  in 
a  manner  closely  resemblinq  the  processing 
normally  performed  iri  the  RWS.  The  signal 
processing  software  uses  the  real  threat 
library  to  Identify,  classify,  and  prior¬ 
itize  the  receiver  inputs.  It  then  gener¬ 
ates  and  updates  the  display  at  the 
trainee's  station.  The  same  display  is 
also  presented  at  the  instructor 's  console 
so  he  can  monitor  and  evaluate  the 
trainee's  performance.  Audio  as  well  as 
visual  displays  are  appropriately  gener¬ 
ated  and  kept  refreshed. 

In  a  traininq  system  which  has  an  in¬ 
herent  ECM  capability  and/or  is  concerned 
with  ECM  effects,  an  "ECM  effects  model" 
can  provide  input  to  the  "effects  genera¬ 
tor"  to  model  the  reaction  to  on-board  or 
off-board  ECM  by  individual  emitters 
(e.g.,  lose  track,  change  frequency)  and 


by  the  RWS  (e.g.,  noise,  lookthrouqh  con¬ 
trol  signals,  support  requests).  Thus, 
reactions  due  to  on-board  and  off-board 
ECM  equipment  can  be  accommodated. 

As  seen  in  figure  4,  the  Instructor's 
control  station  can  control  the  modeling 
in  a  number  of  areas  such  as  emitter  sta¬ 
tus  and  position,  special  propagation  ef¬ 
fects  to  be  included,  special  effects  to 
be  incorporated  such  as  equipment  fail¬ 
ures,  and  EW  effects  to  be  considered  on 
the  emitters. 

Though  not  explicitly  shown  on  the 
diagram,  a  number  of  siqnal  and  data  flows 
are  routed  back  to  the  Instructor;  namely, 
the  audio  and  visual  displays  to  the 
trainee  as  well  as  identification  of 
trainee  action.  Furthermore,  the  system 
provides  for  recording  and  playback  of  the 
threat  environment,  displays  to  the  opera¬ 
tor,  and  operator  actions  which  take  place 
during  a  training  exercise. 

In  addition  to  the  integrated  mode  of 
operation  within  a  flight  trainer,  there 
are  two  stand-alone  modes  of  operation. 

In  one  mode,  the  RWST  can  be  used  as  a 
classroom  trainer  where  an  instructor  can 
demonstrate  most  modes  of  RWS  operation 
against  various  threat  scenarios.  Usinq 
previous  recordings  as  demonstration  aids, 
the  instructor  can  play  back  threat  sce¬ 
nario,  aircraft  position  and  attitude,  and 
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RWS  control  switch  positions.  The  re¬ 
corded  data  would  be  retrieved  from  the 
mass  storage  device  and  input  to  the  "RWS 
siqnal  processing  model"  to  provide  the 
real-time  playback  function. 

The  second  stand-alone  mode  of  RWST 
operation  concerns  Itself  with  the  prepar¬ 
ation  of  traininq  scenarios  and  reprogram- 
minq  of  simulated  equipment  or  special  ef¬ 
fects.  The  purpose  of  the  emitter  model 
scenario  preparation  is  to  allow  the 
placement  of  emitters  and  basic  chanqes  to 
their  performance  characteristics  prior  to 
actual  dynamic  flight  training.  Having 
reprogramming  capability  allows  for 
changes  in  the  operational  aircraft  which 
have  an  effect  on  the  RWS,  to  be  incorpo¬ 
rated  into  the  RWST. 

DESIGN  CONFIGURATION 

Figure  5  represents  the  design  con¬ 
figuration  being  developed  at  ATI.  The 
design  incorporates  a  general  purpose 
"Master  Computer"  and  a  special -purpose 
high-speed  "simulation  processor." 

The  following  functions  are  contained 
in  the  "master  computer:" 

•  Emitter  model 

o  Signal  selection  model 

•  Propagation  effects  model 

•  Aircraft  dynamics  model 

•  Instructor  control  function 


•  Interface  to  flight  simulation 
computer,  radar  land  mass 
simulator,  and  visual  display 
system  computer 

•  Off-line  threat  scenario  genera 
tlon 

The  "simulation  processor"  Includes 
the  following  functions: 

•  Antenna/receiver  modeling 

•  Effects  generator 

•  Real-time  interface  to  the  RWS 

•  ECM  effects 

In  addition  to  the  above  simulation 
functions,  trainer  system  built-in  test 
features  are  included  into  the  design  to 
ensure  daily  operational  readiness  and 
rapid  fault  isolation  in  the  event  of  an 
equipment  failure. 

CONCLUSIONS 

This  new  digital  stimulation  approach 
to  RWST  simulation  is  under  development  at 
ATI.  Detailed  analysis  is  being  performed 
in  the  area  of  data  storage  and  throughput 
requirements  for  various  threat  scenarios, 
threat  parameters,  siqnal  densities  and 
jamming  lookthrough  timings.  At  present, 
the  digital  stimulation  approach  is  quite 
viable  and  offers  several  advantages  over 
prior  approaches.  Table  1  shows  a  compar¬ 
ison  of  the  digital  stimulation  approach 
versus  prior  approaches. 


Table  1. 


Comparison  of  Alternative  Approaches 


RF  Video  Dinital  Total  Software 


Comparison  Item 

Stimulation 

St imul at  ion 

Stimulation 

Simulation 

Difficulty  to 
real istical ly 
simulate  all 
operating  condi¬ 
tions  includinq 

RWS  anomalies 

Highest 

Hiqh 

Moderate 

Hiqh 

Difficulty  of 
attaining  exact 
reoroducibi 1 ity 
of  a  qiven  train- 
ina  scenario 

Highest 

High 

Low 

Lowest 

tlodate  difficulty 
for  chanqes  in 

-  RWS  antennas 

-  RMS  receivers 

-  RWS  diqitizer 

-  RWS  analyzer 

Moderate 

Lowest 

Low 

Low 

Moderate 

Moderate 

Low 

Low 

Moderate 
Moder  ate 
Hiqher 

Low 

Moderate 

Moderate 

Higher 

Hiqhest 

-  RWS  emitter  library 

-  CCM  equipments 

-  Threat  model 

Low 

Highest 

Highest 

Low 

High 

High 

Low 

Low 

Moderate 

Highest 

Low 

Lowest 

Difficulty  in  simulat¬ 
ion  RWS  malfunctions 

Highest 

High 

Low 

Lowest 

Failure  rate 

Highest 

High 

Low 

Lowest 

Need  for  daily 
calibration 

H  iqhest 

Moderate 

Low 

Low 

Need  for  special 
suoonrt  equipment 

Highest 

High 

Low 

Low 

Added  cost  for 
multiple  trainee 
station  simulator 

Highest 

Hiqh 

Low 

Lowest 

Cost  of  expandinn 
threat  environment 

Highest. 

Hiqh 

Low 

Low 

Time  to  repair 

Highest. 

Hiqh 

Low 

Low 

Snare  requirements 

Highest 

Moderate 

Low 

Lowest 

FMI  prohlems 

Highest 

High 

Low 

Low 

Facility  requirements 

Highest 

High 

Low 

Low 

Level  of  training 
required  to  ooerate 
and  maintain 

Hiqhest 

High 

Low 

Low 

Non-recurr inq  cost 
(excluding  software 
development ) 

High 

Moder  ate 

Low 

Moder  ate 

Software  development 
cost 

Moderate 

Moderate 

Hiqh 

Highest 

Recur r inq  cost 
(procurement.) 

Highest 

High 

Low 

Lowest 

Life  cvcle  cost 

Hiqhest 

Hiqh 

Low 

Low 

Cost  oer  degree 
of  realism 

High 

Moderate 

Lowest 

Moderate 

327 


■  Ml-  « 

«  • 


MdBafaiaiiiJtaBi  --*—••  • — 


Thp  RF  stimulation  approach  has  an 
advantage  in  the  ability  to  update  future 
changes  in  the  operational  hardware  with 
minimal  effort  and  effect,  on  traininq 
schedules.  On  the  other  hand,  the  RF 
stimulation  approach  requires  more  spe¬ 
cial-purpose  hardware  than  the  other  ap¬ 
proaches.  This  rpsults  in  loqistics,  re¬ 
liability  and  maintainability  orohlems, 
and  will  qenerally  lead  to  a  hiqh  life  cy¬ 
cle  cost. 

The  advantaqes  of  the  video  stimula¬ 
tion  approach  with  reqards  to  RWR  equip¬ 
ment  updates  are  similar  to  the  RF  stimu¬ 
lation  approach;  however,  future  chanqes 
to  the  RWS  receive  would  require  reoro- 
qrarnminq,  The  requited  hardware  for  video 
stimulation  is  sionif icantly  less  than  for 
RF  stimulation,  thus  imntovinq  the  relia¬ 
bility,  maintainability  and  lifp  cycle 
costs. 

The  software  simulation  approach  uses 
the  least  amount  of  specialized  hardware, 
and  essentially  consists  of  off-the-shelf 
qenet al-puronse  computet  components.  q°t- 
tei  reliability  and  hardware  maintainabil¬ 
ity,  alonq  with  a  low  life  cycle  cost,  are 
the  hiqqest  advantaqes  of  this  annroach. 


However,  the  software  simulation  approach 
has  hiqher  software  development  costs  and 
imposes  additional  time  delays  and  costs 
to  accommodate  RWS  equipment  updates. 

The  diqital  stimulation  approach 
shares  most  of  the  advantaqes  of  the  video 
stimulation  approach  and  the  software  sim¬ 
ulation  approach,  and  practically  none  of 
the  disadvantaqes.  Updates  to  most  of  the 
RWS  components  can  he  made  with  minimal 
effort  and  delay  to  traininq  operations. 
Reinq  computer  based,  the  peculiar 
reliability  and  maintainability  problems 
associated  with  RF  generators,  pulse  and 
scan  qenerators  are  minimized. 

In  summary,  this  new  concept  in  RWR 
simulation  shows  excellent  promise.  The 
qoal  of  hiqhiv  realistic  simulation  at  a 
low  life  cycle  cost,  with  a  flexihle  hard¬ 
ware/software  conf iquration  and  develop¬ 
ment  schedule,  is  achievable.  In  addi¬ 
tion,  the  trend  in  operational  EW  and  avi¬ 
onics  equipment  has  been  oroqressinq  from 
analoq-to-diqital  processing.  In  the 
world  of  trainers  and  simulators,  the 
trend  has  qone  from  analoq-to-diqital  com¬ 
puters.  There  is  no  reason  to  believe 
that  RWS  simulators  should  not  follow  this 
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ABSTRACT 

There  has  been  a  growing  interest  in  the  use  of  Airborne  Weather  Radar 
simulation  from  both  military  and  commercial  operators  of  transport  aircraft. 
This  interest,  in  part,  stems  from  the  need  to  train  aircrews  to  fly  through 
weather  conditions,  which  at  one  time  would  have  been  given  a  "wide  berth", 
without  sacrificing  safety  margins.  This  change  in  requirement  has  arisen  due 
to  escalating  fuel  costs. 

To  date,  the  cost  of  full  radar  simulation  has  restricted  its  use  to  the 
more  sophisticated  aircraft  simulators.  The  Link-Miles  Division  of  Singer  has 
developed  a  low  cost  or  simple  Weather  Radar  Simulator  for  use  on  transport 
aircraft  simulators.  This  paper  describes  the  simulation  techniques  adopted  and 
shows  how  recent  advances  in  semi-conductor  technology  have  been  incorporated  to 
produce  a  system  capable  of  being  fully  integrated  into  an  aircraft  simulator. 


INTRODUCTION 

The  evaluation  of  the  picture  as  it 
appears  on  the  display  of  a  weather  radar 
indicator  is  a  skill  that  increases  with 
experience.  The  information  the  picture 
yields  when  interpreted  correctly  can  mean 
a  significant  reduction  in  operating  costs, 
and  of  even  greater  importance  make  a  major 
contribution  to  flight  safety. 
Discrimination  between  safe  and  potentially 
turbulent  areas  in  cloud  formations  early 
enough  during  a  flight  can  avoid 
unnecessary  " dog- legging"  with  the  inherent 
advantages  in  maintainability  schedules  and 
fuel  saving. 

In  a  situation  where  a  complete  detour 
is  impractical  the  penetration  of  weather 
patterns  may  be  required.  Using  a  radar 
simulator,  cloud  pattern  penetration 
techniques  can  be  mastered  in  the  safety  of 
the  training  complex.  Also  the  weather 
situations  displayed  are  under  instructor 
control,  which  means  both  tropical  or 
temperate  zone  weather  systems  modified  by 
seasonal  variations  can  be  presented  at  the 
"flick  of  a  switch".  It  is  this 
flexibility  coupled  with  the  safety  and 
fuel  saving  aspect  which  renders  this 
system  a  useful  tool  in  aircrew  training. 

Despite  the  obvious  advantages  offered 
by  weather  radar  simulation,  in  general 
this  facility  has  not  been  fitted  to 
transport  aircraft  simulators  in  the  past. 
This  is  mainly  attributed  to  the  relatively 
high  capital  outlay  of  such  a  system. 
However,  recent  advances  in  semi-conductor 
and  micro-computer  technology  has  presented 
designers  with  the  opportunity  to  develop 
systems  which  would  have  been  impractical 
or  prohibitively  expensive  a  few  years  ago. 


Two  such  areas  in  which  this  has  been 
apparent  are  in  the  microprocessor  and 
semi-conductor  memory  fields. 

This  paper  will  show  how  these 
technological  advances  have  been  exploited 
to  produce  a  microprocessor  based  low  cost 
system. 


APPROACH 

A  flexible  approach  was  adopted  in 
the  design  of  the  weather  radar  simulator 
so  that  its  use  would  not  be  restricted  to 
a  specific  type  of  aircraft  or  radar 
system.  A  stand-alone  configuration  was 
considered  as  this  would  facilitate 
installation  to  existing  aircraft 
simulators  with  a  minimum  of  modification. 
Thus  the  simulator  would  store  all  cloud 
information  and  perform  those  tasks 
normally  accomplished  by  the  radar 
receiver/transmitter  and  antenna  sub¬ 
systems  . 

Whilst  most  radar  systems  use 
performance  characteristics  to  ARINC  564, 
the  simulator  would  not  be  restricted  to 
these.  Indeed  the  design  described  is 
based  on  an  EKCO  E290  Radar  System  which 
does  not  conform  to  ARINC  564 
characteristics . 


DESCRIPTION  OP  DESIGN 

The  weather  radar  simulator,  a  block 
diagram  of  which  is  illustrated  in  Figure 
1,  is  intended  for  use  with  an  aircraft 
simulator  which  is  assumed  to  be  controlled 
by  a  computer,  referred  to  here  as  the 
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FIGURE  X.  BLOCK  DIAGRAM  -  LOW  COST  WEATHER  RADAR  SIMULATOR 


"host  computer"  and  serves  to  produce  a 
simulated  radar  signal  for  display  on  a 
standard  weather  radar  indicator  mounted  in 
the  cockpit  of  the  aircraft  simulator. 


Input/Output 

The  interfacing  between  the  host 
computer  and  the  weather  radar  simulator  is 
achieved  via  an  input/output  section.  Air¬ 
craft  location  parameters  and  also  control 
data,  such  as  radar  status  information,  are 
fed  from  the  host  computer  via  a  parallel 
data  path .  The  data  are  received  and 
distributed  via  output  ports  to  other 
sections  of  the  weather  radar  simulator 
under  control  of  a  16  bit  microprocessor. 
The  microprocessor  supervises  the  input  of 
each  data  word  and  performs  a  software 
parity  check  to  verify  data  validity.  It 
then  controls  distribution  of  the  data  to 
the  relevant  output  port  of  the  input/out¬ 
put  section.  It  also  includes  provision 
for  manipulation  of  the  incoming  data  so  as 
to  modify  the  aircraft  position  information 
to  give  the  impression  of  a  dynamic  weather 
system. 

Thus ,  although  the  embodiment 
described  employs  a  store  with  data 
representing  fixed  cloud  formations,  cloud 
movement  can  be  simulated  by  applying  a 
progressive  shift  to  the  input  aircraft 
position  information. 


Beam  Position  Processor 

Aircraft  location  parameters  are  fed 
from  the  input/output  section  via  an  out¬ 
put  port  to  the  beam  position  processor. 
This  section  performs  three  main  tasks 

(a)  system  timing. 

(b)  generation  of  simulated  antenna 


azimuth  position  data. 

( c)  calculation  of  the  X-Y  co¬ 
ordinates  of  points  on  the  radar 
beam  path. 

System  timing  is  derived  from  an 
internal  crystal  oscillator  from  which  are 
obtained  pulses  at  a  pulse  repetition 
frequency  (PRF)  of  400Hz. ,  which  is  the 
frequency  typically  used  in  the  E290  Radar. 

The  beam  position  processor  generates 
angles,  representing  antenna  azimuth 
position  over  a  range  ±90°  of  aircraft 
heading,  in  the  form  of  a  12  bit  data  word. 
This  is  output  to  the  video  processing 
section,  which  will  be  described  further 
below.  Assuming,  as  above,  a  400Hz  PRF 
and  a  typical  120°/sec.  scan  rate,  the 
antenna  azimuth  rotation  between  successive 
radar  pulses  is  0.3°. 

Using  the  X-Y  coordinates  of  the  air¬ 
craft  position  input  from  the  host 
computer,  along  with  the  simulated  antenna 
position  information  generated  as  described 
above,  the  X-Y  coordinates  of  the  radar 
beam  at  successive  positions  over  the  range 
under  consideration  are  derived.  Based  on 
the  E290  Radar  performance  characteristics 
a  range  of  175  nautical  miles  is  covered, 
and  to  achieve  maximum  resolution  at  short 
range  the  simulated  range  increments  are 
non-linear.  Thus  the  beam  position  for 
range  values  between  0  and  20  n. miles  is 
based  on  a  0.25  n. miles  increment,  while 
values  between  20-50  n. miles  are  based  on 
a  0.5  n. miles  increment  and  for  the  50-175 
n. miles  a  1.00  n. miles  increment  is  used. 
This  produces  a  total  of  265  range 
increments  per  scan.  With  a  PRF  of  400Hz 
this  gives  a  radar  beam  position  update 
rate  of  9.4(iS.  The  radar  beam  position  is 
calculated  for  each  of  the  265  range 
increments  for  every  0.3°  azimuth  scan. 
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To  take  into  account  the  beam  width 
characteristic?  of  the  antenna  main  lobe, 
two  other  beam  positions  are  calculated  at 
angles  il.75°  of  the  boresite. 

The  X-Y  coordinates  are  calculated 
usings  - 

X  -  Rsin(e+4) +xa^c  n. miles 
Y  =  Rcos<0+$)+Ya^o  n. miles 

where R  =  Range 

0  =  aircraft  heading  angle 

4  =  antenna  pointing  angle 

X  .  ,  Y„.  =  X-Y  coordinates  of  aircraft 

a/c  a/c  location. 

Therefore,  for  each  9.4pS  period,  three 
values  of  X-Y  coordinates  are  obtained. 


Cloud  Store 

The  X  and  Y  beam  coordinates  are  used 
to  read  out  information  from  a  cloud  store. 
Basically  the  store  contains  a  cloud  map 
in  which  the  area  covered  is  divided  into 
cells  in  which  information  is  stored  of 
cloud  intensity,  together  with  base  and 
ceiling  heights.  The  manner  in  which  this 
information  is  stored  will  now  be 


discussed  in  more  detail.  The  cloud  store 
contains  three  memories  each  with  a 
capacity  of  64K  bytes  provided  by  thirty- 
two  2K  x  8  erasable  programmable  read  only 
memories  (EPROMS) .  The  first  memory,  the 
scenario  memory,  stores  up  to  16  scenarios. 
Each  scenario  occupies  a  memory  space  of 
4K  x  8,  representing  4096  cloud  pattern 
locations  on  a  64  x  64  matrix.  Each  memory 
location  contains  an  8  bit  data  word  which 
is  a  code  specifying  which  of  a  pre¬ 
determined  number  of  possible  cloud  patterns 
occurs  in  that  particular  cloud  pattern 
location.  The  six  most  significant  bits  of 
each  of  the  X  and  Y  coordinates  from  the 
beam  position  processor  provide  a  12  bit 
address  to  the  scenario  memory.  The 
remaining  4  bits,  for  selection  of  the 
desired  scenario,  are  input  from  an  output 
port  on  the  input/output  section.  The 
selection  originating  from  the  instructor's 
facility . 

The  8  bit  data  format  used  in  the 
scenario  memory  is  in  principle  capable  of 
selecting  from  up  to  a  maximum  of  256 
patterns,  but  only  16  cloud  patterns  have 
been  provided  for  this  particular 
application.  The  cloud  patterns  are  stored 
in  the  second  and  third  memories.  Each 
cloud  pattern  occupies  a  memory  space  of 
4K  x  16  and  represents  an  area  of  sky 
32  x  32  n. miles  to  a  resolution  of  0.5 
n. miles  square.  Therefore,  a  cloud  pattern 
consists  of  4096  cloud  cells  on  a  64  x  64 
matrix.  The  cloud  pattern  memories  receive 


FIGURE  2.  CLOUD  STORE  CONFIGURATION 
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a  12  bit  address  from  the  beam  position 
processor  (i.e.  the  least  significant  bits 
of  the  X  and  Y  beam  coordinates) ,  and  4 
bits  from  the  scenario  memory.  The  16  bit 
data  format  used  in  the  cloud  pattern 
memory  contains  information  on  cloud  cell 
intensity,  and  cloud  base  and  ceiling 
heights.  This  data  is  formatted  as:- 

3  bits  -  cloud  intensity  (8  levels) 

8  bits  -  ceiling  height  (in  units  of 

1,000  ft.) 

8  bits  -  base  height  (in  units  of 

1,000  ft.) 

1  bit  -  unused 

This  arrangement  enables  a  three- 
dimensional  cloud  map  to  be  formed  covering 
an  area  of  2048  x  2048  n. miles  x  64,000ft. 

A  scenario  is  configured  by  coding  any  of 
the  16  cloud  patterns  into  each  of  the  4096 
locations  in  the  scenario  memory,  as 
illustrated  in  Figure  2.  It  will  be 
appreciated  that  the  two  tier  memory 
arrangement  described  represents  a  very 
substantial  reduction  in  memory  require¬ 
ments  compared  with  that  which  would  be 
required  if  a  separate  memory  location  were 
provided  for  each  cell  of  the  total  area  to 
be  covered. 

It  will  be  appreciated  that  the  cloud 
store  arrangement  described  does  not  admit 
the  possibility  of  two  or  more  discontinuous 
sections  of  cloud  within  a  single  cloud 
cell.  Obviously  the  cloud  pattern  memory 
capacity  could  be  expanded  to  permit  the 
inclusion  of  more  than  one  set  of  intensity 
and  height  data  per  cell  to  admit  the 
possibility  of  overlaying  cloud  formations. 


Altitude  and  Intensity  Processor 

The  cloud  information  from  the  cloud 
store  is  fed  to  the  altitude  and  intensity 
processor.  The  first  task  of  this  section 
is  to  establish  whether  the  radar  beam 
height  lies  within  the  altitude  range 
specified  by  the  base  and  ceiling  heights 
of  the  cloud  cell  under  consideration. 

The  radar  beam  height  is  given  by  the 
aircraft  height  modified  by  the  antenna 
tilt  components.  The  latter  is  given  by:- 

ztilt  =  k.Rsin(a)  (feet) 

where  R  =  range  of  beam  (n. miles) 

a  =  antenna  tilt  angle  (degrees) 
k  =  conversion  constant  (feet/n. miles) 

Thus  the  altitude  processor  receives 
aircraft  height  information  from  the  host 
computer  via  a  output  port  of  the  input/ 
output  section,  range  information  from  the 
beam  position  processor,  and  antenna  tilt 
angle  as  selected  by  the  user  at  the  radar 
control  panel.  The  latter  can  be  input 


directly  from  the  control  panel,  or  in 
this  application  via  the  aircraft  simulator 
linkage,  to  be  output  eventually  by  the 
host  computer.  Using  these  data  inputs 
the  height  of  the  instantaneous  beam 
position  is  calculated.  This  height  is 
compared  with  the  base  and  ceiling  heights 
of  the  cloud  cell  under  consideration 
(from  the  cloud  store) .  If  the  beam  falls 
within  these  heights,  the  cloud  cell 
intensity  data  from  the  cloud  store  is 
routed  to  the  intensity  processor. 

The  intensity  processor  serves  to 
determine  the  intensity  of  the  "paint"  on 
the  radar  indicator.  This  can  be  achieved 
by  using  the  standard  radar  equation: - 

pr  =  Pt^e0 
(4s)  ^R1* 

where Pr  =  reflective  power 

Pt  =  transmitted  power 

G  =  maximum  radiation  power  gain 

Ag  =  effective  receiving  antenna  area 

o  =  echo  range  of  target 

r  =  range 

However,  radar  equation  computation 
is  avoided  by  following  a  simplified 
procedure.  The  intensities  of  all  the 
cloud  cells  the  beam  has  passed  through 
before  reading  the  location  under 
consideration  are  accumulated,  and  this 
accumulated  Intensity  along  with  the 
range  are  used  to  address  a  look-up  table, 
(e.g.  a  read  only  memory)  containing 
empirically  derived  weighting  factors. 

In  this  way  a  radar  return  intensity 
signal  is  obtained  as  an  8  bit  data  word. 
Although  this  involves  a  degree  of 
approximation,  it  has  been  found  in 
practice  that  the  resultant  signals  are 
sufficiently  accurate  for  the  purpose, 
particularly  when  (as  is  usually  the  case) 
the  raw  radar  returns  are  processed  to  a 
degree  by  the  indicator  unit  before 
actually  being  displayed  on  the  cathode 
tube.  This  is  especially  valid  in  the 
case  of  colour  displays  which  are  now 
replacing  the  older  direct  view  storage 
tube  (DVST)  type  displays. 

Normally  the  data  corresponding  to 
the  centre  (boresite)  position  is  used 
unless  this  does  not  encounter  any  cloud, 
in  which  case  the  intensities  at  either 
side  of  the  main  lobe  are  considered. 

The  intensity  values  are  computed 
sequentially  for  the  265  range  increments, 
and  owing  to  the  non-linear  increments 
used,  the  resultant  signal  has  a  non¬ 
linear  range/time  relationship.  For  use 
with  a  conventional  radar  display  unit 
these  signals  must  be  linearised  and  this 
is  achieved  by  clocking  the  intensity 
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value  for  each  range  increment  into  a  road/ 
write  memory  at  one  rate  and  clocking  them 
out  at  another  rate  synchronised  to  the 
display.  To  maintain  continuity  of  data 
flow,  two  memories  arc  used  which  are 
interchanged  after  each  radar  scan. 

Finally,  the  altitude  intensity  processor 
converts  the  calculate  8  bit  intensity 
word  for  each  range  increment  into  an 
analogue  signal. 


Video  Piocessot 

The  final  stage  of  the  weather  radar 
simulator  comprises  a  video  processor  and 
deflection  section.  The  video  processor 
serves  to  receive  the  video  signal  from 
the  altitude  and  intensity  processor  and 
to  add  a  number  of  special  effects  required 
to  add  realism  to  the  radar  "paint",  and 
these  are  mixed  with  the  video  signals. 

The  special  effects  include :- 

(a)  spurious  noise 

(b)  height  ring 

( c)  ground  Leturns 

(d)  11. F.  spoking 

(e)  radome  icing 

The  composite  radar  signal  is 
amplified  and  output  via  a  coaxial  line  to 
the  radar  indicator.  In  addition  a  pre¬ 
pulse  synchronisation  signal,  derived  from 
the  beam  position  processor,  is  also 
amplified  and  output  to  the  radar  indicator. 

The  deflection  section  produces 
indicator  deflection  signals  corresponding 
to  the  antenna  pointing  angle,  the  latter 
being  output  as  a  12  bit  word  from  the 


beam  position  processor.  These  are 
converted  into  signals  which  would  normally 
be  output  by  a  synchro- resolver  mounted  on 
the  antenna  mechanism  of  an  actual  radar 
system. 


SUMMARY 

The  weather  radar  system  described 
was  developed  to  meet  tile  requirements  of 
one  particular  customer.  There  are  several 
enhancements  which,  although  not  taken  up 
by  that  customer,  can  be  added  to  this 
system.  These  include :- 

(a)  Storm  cell  growth  and  decay:  where 
storm  cells  appear  to  move  in  relation 
to  each  other,  whilst  increasing  and 
decreasing  in  size. 

(b)  Windsheer  Vectors:  whilst  the  present 
system  is  capable  of  operating  on  the 
entire  weather  mao  using  a  common 
windsheer  vector,  this  enhancement 
will  cause  movement  of  the  weather- 
map  using  several  windsheer  vectors 
differing  in  both  height  and 
direction . 

The  following  photographs  were  taken 
of  an  E 290  radar  indicator  driven  by  the 
radar  simulator.  Figure  3  illustrates  a 
squall-line  ahead  and  just  to  starboard  of 
the  aircraft  track.  Figures  4a-4f  are  a 
sequence  of  six  frames  showing  the 
negotiation  of  a  complex  weather  system 
generated  by  the  radar  simulator. 
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ABSTRACT 

The  correct  use  of  modern  radar  in  flight  and  attack,  aircraft  is  one  of  the  main 
tasks  of  the  pilot  and  copilot.  This  means  that  training  in  radar  operation  and 
image  interpretation  is  one  of  the  most  important  training  aspects  in  a  full 
mission  simulator.  Realistic  simulation  of  modern  radars  is  essential  to  achieving 
the  required  training. 

The  state-of-the-art  in  micro-electronics ,  data  processing,  computer  graphics  and 
other  relevant  areas  make  it  possible  to  generate  all  necessary  radar  signals  in 
real-time  using  completely  synthetic  video  generation  methods.  Based  on  digital 
data  bases  and  the  appropriate  aircraft  and  radar  parameters,  terrain  and  culture 
profiles  are  generated.  From  these  profiles,  radar  echoes  simulating  Ground  Mapping 
Radar,  Terrain  Following  Radar  and  the  Radar  Altimeter  are  synthesized.  Typical 
radar  effects  such  as  range  attenuation,  shadowing,  far-shore  enhancement,  and 
pulse  stretching  are  accounted  for  along  with  the  antenna,  transmitter,  and  receiver 
characteristics  of  the  actual  radar  in  all  operating  modes. 

The  radar  simulation  is  performed  in  real-time,  with  all  computations  up-dated  at 
a  rate  consistent  with  the  radar  characteristics  and  the  fiight  control  system  of 
the  aircraft.  The  use  cf  a  high  up-date  rate  allows  all  training  missions  t n  be 
performed  with  full  freedom  of  manoeuvring  in  the  simulator. 

This  paper  describes  the  performance  characteristics  of  a  Digital  Radar  Landmass 
Simulation  System  (DRLMS),  developed  for  the  TORNADO  Flight  and  Tactics  Simulator, 
with  an  emphasis  on  its  training  role. 


INTRODUCTION 

The  European  Multi  Role  Combat  Aircraft 
Tornado  is  equipped  with  one  of  the  most  modern 
and  sophisticated  Radar  systems.  The  system  has 
been  developed  by  Texas  Instruments.  For  the 
Tornado  Operational  Flight  and  Tact’cs  Simulator 
the  German  Air  Force  and  Navy  as  well  as  the 
Italian  Air  Force  have  decided  to  adopt  a  Digital 
Radar  Landmass  Simulation  System  developed  by 
Messerschmitt-Boelkow-Blohm,  Dynamics  Division 
with  certain  know  how  inputs  from  General  Electric, 
SCSD  Division,  Daytona  Beach,  Florida. 

In  the  meantime  the  first  two  systems  have  been 
accepted  by  the  customer. 


RADAR  SIMULATION  OF  MRCA 

,  The  primary  function  of  the  Multi  Role 
Combat  Aircraft  (MRCA)  Digital  Radar  Landmass 
Simulation  (DRLMS)  is  to  provide  realistic  simu¬ 
lated  radar  display  video  for  the  ground  mappinq 
radar  (GMR)  and  terrain  following  radar  (TFR) . 

The  simulated  radar  video  is  subsequently  pre¬ 
sented  on  the  operation/maintenance  station  ra¬ 
dar  display  and  the  K.TCA  Operational  Flight  and 
Tactics  Simulator  (OFTS)  simulator  system  dis¬ 
plays.  Simulated  video  is  also  generated  for  pre¬ 
sentation  when  either  the  contour  mapping  on- 
ooresight  (CMO)  or  target  height  finding  (THF) 
mode  of  the  GMR  is  selected.  As  selected  by  mode, 
the  content  of  the  simulated  video  includes  a 
predetermined  combination  of  terrain  and  culture 
returns,  environmental  and  mission  effects, 
jamming  and  noise  interference,  air  and  seaborne 
target  skin  paint  returns,  and  beacon  code  video. 


The  format  of  the  DRLMS  simulated  video  output 
is  compatible  with  the  MRCA-OFTS  interface  for 
each  of  the  varius  operating  modes  of  the  MRCA 
nose  ground  map  radar  set  and  the  terrain  follow¬ 
ing  radar. 

Other  functions  c*  the  MRCA  DRLMS  subsystem 
include  the  altimeter  radar,  the  occulting  of 
fixed  site  and  moving  platform  threats,  in¬ 
corporation  of  mission  and  environment  effects, 
generation  of  thunderstorm  or  chaff  backscatter 
.-.nd  data  base  up-dating . 

Principal  features  of  the  MRCA  DRLMS  subsystem 
include  European  gaming  areas,  accurate  terrain 
and  planimetry  data  for  each  radar  pulse  width 
and  on-line  data  storage  capacity  for  a  minimum 
of  5,000.000  square  kilometers  (1,500.000  square 
nautical  miles). 


MRCA-DRLMS 

The  Digital  Radar  Landmass  Simulator  (DRLMS) 
is  a  subsystem  of  t.he  Multi  Role  Combat  Aircraft 
(MRCA)  Operational  Flight  and  Tactics  Simulator 
(OFTS).  The  DRLMS  in  the  OFTS  is  shown  in  Figure  1 

Interface  data  from  the  MRCA-OFTS  that  define  the 
current  aircraft  position,  radar  control  functions 
and  the  threat,  weapon,  beacon  and  jamming  re¬ 
quirements  of  the  mission  are  supplied  to  the 
shared  core  memory  nortion  of  the  MRCA-DRLMS 
computer.  Interface  data  from  MRCA-DRLMS  that 
define  the  GMR-Video  and  TFR-Profil  are  supplied 
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Figure  1:  DRLMS  in  the  OFTS 


date  geometry.  In  addition  to  the  region  data, 
the  real-time  data  includes  dynamic  target  and 
occulting  data  from  the  OFTS-computer.  These 
data  are  sent  by  the  DRLMS-computer  to  the  moving 
target  memory.  The  moving  target  memory  is  also 
holding  weather  and  chaff  data  as  established  by 
the  OFTS  instructor  selection. 

The  data  base  processor  includes  region  selector, 
segment  stripper  and  segment  memories.  Segment 
data  are  selected  from  the  region  memory  by  the 
region  selector  in  the  region  controller.  Radar 
return  response  data  for  point  targets  in  the  word 
format  are  target  identity,  directivity,  reflec¬ 
tivity,  and  target  width.  Radar  return  response 
data  for  c.ulture  edge  word  contain  reflectivity 
and  edge  type  identity.  Terrain  edge  words  carry 
no  reflectivity.  The  weather  code  replaces  re¬ 
flectivity  for  weather  and  chaff  edges.  For  beacon, 
dynamic  target  data,  a  beacon  code  is  included. 

For  dynamic  (or  point)  targets,  the  return  is  de¬ 
pendent  on  the  cross-sectional  area  code. 

Figure  5  illustrates  the  home  region  of  the  air¬ 
craft  and  the  surrounding  8  regions  that  are  al¬ 
ways  read  by  the  region  selector.  The  reminder  of 
the  regions  selected  grossly  approximate  the  seg¬ 
ment  width  to  be  stripped  (©52)’ 

The  segment  stripper  tests  reduce  the  data  to  that 
pertinent  to  the  segment  width  (©  So)  f°r  storage 
in  the  segment  memories. 


to  the  simulator  cockpit,  altimeter-  and  threat 
occult-data  are  supplied  to  the  OFTS  main  computer. 

The  physical  system  configuration  is  shown  in  Fi¬ 
gure  2.  The  major  equipment  items  that  comprise 
the  MRCA- DRLMS  are  the  DRLMS-Computer  system  (CPU, 
tape,  3  discs,  ternrnals  and  printer),  the  DRLMS- 
radar  processor  with  the  power  control  and  the 
operator/maintenance  station  (operator /maintenance 
console  with  a  summagraphics  digitizer  tablet, 
graphic/alphanumeric  terminal  and  hard  copy  unit). 

The  system  function  block  diagram  (Figure  3)  is 
helpful  for  explanation  of  the  DRLMS  function. 
Figure  3  shows  the  system  block  diagram  with  the 
principal  data  flow  represented  by  solid  lines, 
and  the  basic  control  functions  represented  by 
broken  lines. 

Interface  data  from  the  MRCA-OFTS  that  define 
the  current  aircraft  position  and  radar  control 
functions  are  supplied  to  the  DRLMS  computer. 
Regions  of  real-time  digital  data  are  selected  as 
a  function  of  aircraft  position  and  read  from  one 
of  three  300  megabyte  disc  units. 

The  selected  region  data  are  stored  in  the  DRLMS 
region  memory.  The  regions  stored  there  are  for 
the  long-,  medium-  and  short-range  data  bases. 

A  square  set  of  region  is  selected  about  the  air¬ 
craft  home  region  at  the  mission  initial  point 
for  each  data  base. 

Thereafter,  the  active  real-time  data  base  is  up¬ 
dated  by  deleting  regions  out  of  range  because 
of  aircraft  movement  and  by  adding  new  regions 
which  are  presently  within  range  of  the  radar. 

This  is  accomplished  each  time  the  aircraft  crosses 
a  region  boundary  and  thereby  enters  a  new  home 
region.  Refer  to  Figure  4  for  typical  region  up- 


Th"  data  word  type  and  X/Y  position  is  used  by  the 
Segment  stripper  in  determining  which  edges  and 
targets  (points)  are  to  be  selected  for  each  si¬ 
mulated  antenna  scan  segment.  All  GMR  and  ALT  edge 
or  point  words,  passing  the  segment  tests,  are 
stored  in  a  common  active  edge  ping-pong  memory. 

A  separate  memory  is  maintained  for  TFR  or  TOP 
data.  Each  unique  memory  segment  of  data  contains 
the  data  for  the  next  set  of  scan  lines. 

The  data  sorter  provides  three  functions  (inter¬ 
section  processor,  intercept  orderer,  and  priori¬ 
ty  resolver  and  data  profile  storage).  The  inter¬ 
section  processor  determines  points  of  intersection 
on  the  process  sweep  line.  In  so  doing,  it  con¬ 
verts  the  X,  Y,  Z  cartesian  vertices  received 
from  the  second  memories  to  range  (R)  and  Z  coordi¬ 
nates  at  the  process  sweep  line  angle  of  the  in¬ 
tersection  processor.  These  converted  data  are 
ordered  from  near  range  to  far  range  by  the  inter¬ 
cept  orderer.  Then,  the  data  culture  is  resolved 
by  priority  number  where  conflicts  exists  for 
which  data  are  to  reside  in  a  particular  range 
process  element  of  the  process  sweep  line.  Then 
resolved  data  are  stored  except  for  weather  and 
chaff.  These  data  are  sent  direct  to  the  weather 
processor.  The  terrain  data  are  processed  and 
stored  and  require  no  priority  resolution  because 
terrain  data  carries  no  reflectance. 

The  power  processor  includes  natural  effects 
processor  (NEP),  weather  processor  (WEP),  alti¬ 
meter  processor  (ALT),  terrain  following  radar 
processor  (TFR)  and  target  occulting  processor 
(TOP).  The  ground  range  ordered  terrain  profile 
and  culture  profile  are  output  from  the  data  sor¬ 
ter  to  the  natural  effects  processor.  The  natural 
effects  processor  determines  the  slant  range  and 
angle  of  incidence  relative  tothe  radar  antenna 
(aircraft  position)  above  che  earth  for  each 
successive  simulation  pointing  angle  of  the  radar 
antenna.  Using  this  data  and  transmitter  power, 
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antenna  gain  and  assigned  backscatter  coefficients 
in  the  culture  reflectance  codes  or  weather  data 
codes,  the  natural  effects  function  determines  the 
solution  of  the  radar  return  equation.  Four  ver¬ 
sions  of  this  equation  are  used  to  provide  for 
areal  target  returns,  point  or  line  tarqets  re¬ 
turns,  weather  returns  and  beacon  or  jammer  re¬ 
turns.  Radar  shadow  flags  and  special  effects  such 
as  fare-shore  enhancement,  and  sea  state  are  added. 
These  returns,  in  decibels  (dB),  are  sent  to  the 
radar  effects  processor  along  with  the  slant 
range  (SR). 

The  weather/chaff  processor  provides  weather/ 
chaff  backscatter  return  levels  and  weather 
atmospheric  attenuation  for  all  GMR  modes,  and 
the  ALT  and  TFR  mode  cycles.  The  weather/chaff 
processor  is  not  required  for  TOP  functions. 

For  altimeter  (ALT)  radar  simulation,  the  alti¬ 
meter  radar  return,  simulator  portion  of  the 
natural  effects  processor  tests  are  calculated 
return  level  in  the  360°  altimeter  antenna  field 
of  view  about  the  aircraft  and  saves  only  those 
that  are  above  the  altimeter  radar  receiver  sen¬ 
sitivity  threshold.  A  further  test  saves  the 
first  return  with  the  nearest  slant  range  in  the 
field  of  view.  These  data  are  output  to  the 
OFTS  computer. 

Terrain  following  radar  (TFR)  information  selected 
from  the  medium-range  active  real-time  data  base 
is  furnished  to  TFR  processor  by  the  natural 
effects  processor  (NEP).  These  data  are  subsequent¬ 
ly  slant  range  ordered  and  are  accumulated  for  6 
consecutive  TFR  scan  line  segments  to  obtain  an 
average  power  level  for  the  TFR  antenna  field  of 
view.  Upon  completing  the  sixth  iteration  of  the 
order  and  accumulate  process,  a  TFR  slant  range 
ordered  profile  is  generated  as  a  function  of  the 
accumulated  data,  that  is  output  to  the  OFTS  inter¬ 
face  and  the  video  controller  portion  of  the  video 
calculator. 

The  threat  occult  processor  (TOP)  provides  2 
functions: 

(1)  computes  the  effect  of  j aiming  for  up  to  4 
threats  that  are  identified  as  being  jammers 
and 

(2)  records  the  occult  status  for  up  to  64  threats. 

The  threat  data  includes  the  occult  status  ele¬ 
vation  above  mean  sea  level  and  threat/weapon 
identification  number.  The  TOP  performs  the  ne¬ 
cessary  calculation  to  determine  the  occult  status 
of  each  threat  target.  These  data  are  identified 
by  threat  target  ID  number  from  input  to  the  mo¬ 
ving  target  memory  until  it  is  returned  to  the 
DRLMS  computer.  The  TOP  provides  jamming  infor¬ 
mation  to  the  GMR  video  channel  for  each  GMR  sweep 
output.  Jamming  simulation  data  are  also  provided 
for  each  TFR  sweep. 

The  video  calculator  includes  the  radar  effects 
processor  (REP)  and  the  video  controller  (VC). 

The  radar  effects  processor  (REP)  receives  the 
radar  return  dB  levels  and  slant  range  for  terrain/ 
culture,  dynamic  targets,  weather  and  chaff,  and 
beacon  from  the  natural  effects  processor  (NEP). 

The  beacon  code,  beacon  flag,  glitter  flag,  and 
CMO  flag  are  also  received  from  the  NEP.  The 
terrain/culture  dB  is  slant  range  ordered,  in¬ 


cluding  airborne  target  dB.  Weather  is  indepen¬ 
dently  slant  range  ordered.  The  pulse  width  error 
(or  pulse  width  integration  effects),  horizontal 
beamwidth  integration,  beacon,  jamming,  re¬ 
ceiver  noise,  radar  receiver  characteristics, 
sensitivity  time  constant,  and  conversion  from 
dB  to  volts  are  than  accomplished.  The  output 
is  at  the  operating  bandwidth  resolution  for  dis¬ 
play  output  to  the  video  controller.  Jammer  dB 
levels  and  jammer  type  identifying  flags  are  re¬ 
ceived  by  radar  effects  from  the  target  occul¬ 
ting  processor. 

The  video  controller  (VC)  receives  the  video 
levels  in  volts  and  in  digital  format  from  the 
radar  processor.  The  video  is  digital -to-analog 
(D/A)  converted  and  is  processed  to  output  GMR 
video  to  the  OFTS.  Either  GMR  or  TFR  outputs 
may  be  selected  for  display  on  the  maintenance 
console. 

Functional  control  of  the  DRLMS  is  directed  from 
the  process  controller  (PC).  The  process  con¬ 
troller  function  contains  a  control  oriented 
computer  (CORC).  The  CORC  synchronizes  and  directs 
all  of  the  DRLMS  computer/processor  activities. 

It  accepts  radar  parameter  and  aircraft  data 
from  the  DRLMS  computer.  The  OFTS  computer  pro¬ 
vides  dynamic  control  data  for  the  radar  control 
parameters  to  the  DRLMS  every  55  msec.  This  in¬ 
formation  is  than  distributed  to  the  applicable 
functions  in  DRLMS,  in  a  manner  that  is  trans¬ 
parent  to  the  OFTS  system  operators.  The  CORC  is 
a  iixed  program,  interrupt-driven  device.  Funda¬ 
mental  to  the  pipeline  process  timing  in  the 
DRLMS  is  the  selected  operating  mode  and  pulse 
repetition  frequency  (PRF)  of  the  radar.  Also 
function  of  the  operating  mode  is  the  effective 
antenna  scan  rate.  The  PRF  established  the  time 
between  radar  sweeps  and,  thus,  the  angle  between 
sweeps  for  a  given  PRF  and  scan  rate. 

The  DRLMS  computer/processor  has  two  fundamental 
operating  modes:  On-line  and  off-line.  Selection 
of  the  operating  mode  is  accomplished  by  the 
operator  at  the  DRLMS  operator/maintenance  station. 
Data  base  up-date  and  diagnostic  and  maintenance 
operations  are  conducted  in  the  off-line  operating 
mode.  The  diagnostic  data  base  for  these  opera¬ 
tions  is  stored  on  a  dedicated  portion  of  one  of 
the  300-megabyte  discs.  Control  of  the  DRLMS  sub¬ 
system  for  the  off-line  operating  mode  is  exerci¬ 
sed  from  the  operator/maintenance  console. 

The  method  of  data  base  modelling  is  of  criti¬ 
cal  importance  in  establishing  the  DRLMS  design 
approach. 

The  digital  data  base  is  a  set  of  numerical  data 
that  defines  the  terrain  elevation  and  the  radar 
significant  featu.  as  on  the  earth  surface  for  a 
given  geographical  areal,  which  is  prefered  to 
as  the  gaming  area.  The  realism  possible  from  a 
radar  simulator  is  directly  dependent  on  the 
quality  (or  fidelity)  of  the  digital  data  base, 
for  without  a  valid  discription  of  the  gaming 
area  the  best  of  software  and  hardware  cannot 
create  a  realistic  radar  image. 

The  data  capacity  of  the  DRLMS  for  the  on-line 
data  base  is  sufficient  to  store  at  least  the 
data  of  a  5  million  square  kilometer  gaming  area. 
Out  of  a  1.996  x  1.996  nm  square  an  arbitrarily 
gaming  area  may  be  choosen.  MRCA  DRLMS  real-time 
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radar  data  base  boundaries  are  shown  In  Figure  6. 

3  GAM t NO  AAEA  GAMING  AREA 

TERRAIN  CULTURE 

SOURCE  DATA  (DMA)  SOURCE  DATA  <DMA) 


MAGNETIC  TARE  MAGNETIC  TARE 


300  MEGAOYTE  DISC  RACKS 


RADAR 

INDICATORS 


Figure  6:  Aircraft  360  Degree  Field-of-View  Figure  7:  Radar  Data  Base  Flow 

in  Real-Time  Radar  Data  Base 


Transformation  of  defence  mapping  agency  aero¬ 
nautical  center  ( DMAAC )  terrain  and  planimetry 
source  data  and  merging  of  these  transformed 
data  into  a  single  radar  data  base  file  are 
accomplished  by  the  transformation  program.  The 
transformation  program  is  fully  automatic  imple¬ 
mentation  of  the  terrain  and  planimetry  compres¬ 
sion  technics  described.  Three  digital  data  base 
maps  (DDM)  are  generated,  one  for  each  level  of 
resolution  to  be  simulated.  Each  DDM  defines  both 
the  terrain  elevation  and  radar  reflectivity  for 
the  entire  gaming  area.  Radar  reflectivity  is 
defined  in  terms  of  a  reflectance/ texture  code 
associated  with  each  discrete  radar  point/line 
target  and  each  bounded  (areal)  target.  Target 
reflectivity  codes,  32  for  point/line  targets 
and  32  for  areal  targets,  are  assigned  by  the 
transformation  program  as  a  function  of  a  DMAAC 
feature  identification  codes.  These  feature  iden¬ 
tification  codes  describe  all  features  in  the 
planimetry  file  with  descriptive  information  set 
characterizes  the  feature  in  terms  of  its  surface 
material,  e.g.  matter,  stone/brick, water,  snow/ice, 
trees,  rock. 

The  final  step  in  generation  a  transformed  DD8 '■l 
from  the  DMAAC  source  data  is  to  concentrate  the 
separately  generated  terrain  and  planimetry  files 
into  a  single  DDB  file  for  each  DRUMS  region  of 
the  gaming  area.  This  process  consists  o *  merging 
the  two  files  to  a  single  disc  pack  or  tape,  as 
required.  The  output,  of  the  merge  process  is  the 
transformed  digital  data  base.  Figure  7  summarizes 
the  data  base  data  flow,  processing  and  storage. 


CONCLUSION 

Since  the  Trainings  Simulator  for  Tornado 
is  used  as  a  Full  Mission  Simulator  the  system 
is  equipped  with  a  Compjter  Generated  Image 
Visual  System  (CGIVS)  in  order  to  train  among 
others  weapon  delivery  and  position  up-dating 
by  means  of  the  head-up  display.  Necessary 
correlation  of  information  presented  by  DRLMS 
and  CGIVS  is  achieved  by  common  source  material 
used  to  generate  the  data  bases.  The  source 
material  is  cartographic  information  in  digital 
form  (DMA-format) .  By  means  of  a  data  base 
generation  system  under  development  for  the 
German  services  CGIVS  and  DRLMS  data  bases 
will  be  created. 


+)  DDB  -  digital  data  base 
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ABSTRACT 

Mid-range  trainers  are  designed  to  provide  useful  training  at 
significantly  lower  cost  and  development  time  than  those  for 
full -capability  (full-scale)  trainers.  Mid-range  trainers  are 
appropriate  for  use  prior  to  development  of  full-scale  trainers  and  as 
supplements  to  full-scale  trainers.  With  a  mid-range  trainer, 
trainees  may,  for  example,  practice  operational  procedures,  become 
familiar  with  system  control  locations  and  reaction  times,  and  learn 
to  recognize  and  handle  system  faults.  This  paper  describes  the  B-52 
OAS/CMC  Part  Task  Trainer,  detailing  Its  function  as  both  a  conversion 
trainer  to  provide  the  highest  level  of  simulated  training  while  the 
full-range  trainer  is  being  developed,  and  as  a  “lead  In"  trainer 
before  the  student  moves  to  the  full-range  device.  Capabilities  of 
the  Part  Task  Trainer  are  explained.  Cost  effective  aspects  of  dual 
use  are  discussed.  Design  features  to  provide  lower  Initial  cost  and 
flexibility  for  future  modification  and  expansion,  use  of 
minicomputers,  off-the-shelf  components,  and  modular  structure  are 
detailed. 


INTRODUCTION  and  training  devices  are  undergoing  change 

concurrently  with  changes  In  the  weapon  system  to 
Mid-range  trainers  provide  high  training  meet  training  requirements  and  objectives, 

value  with  lower  cost  and  shorter  development 

time  then  full-range  trainers,  such  as  weapons  In  this  fluid  training  situation,  the 

systems  trainers.  They  can  be  applied  mid-range  device  will  be  used  as  an  Interim 

effectively  in  situations  requiring:  1)  a  device  before  development  of  a  full-range  trainer 

conversion  trainer,  to  train  new  or  unique  such  as  a  Weapon  System  Trainer.  These  devices 

procedures,  tasks,  and  functions  in  the  interim  would  be  designed  to  train  procedures,  task  and 

during  which  the  weapon  system  Is  being  built  or  functions  of  a  new  system  before  moving  to  the 

modified,  or  when  no  full-range  trainer  is  weapon  system  Itself.  This  training  concept 

available;  and  2)  an  Initial  lead  In  trainer,  for  saves  aircraft  hours  needed  for  training  and 

when  the  weapon  system  or  full  range  trainer  is  effectively  trains  lower-level  training  tasks, 

in  complete  operation  but  training  is  required  With  these  devices,  the  students  or  trainees  will 

for  the  user  to  become  skilled  at  lower-level  become  familiar  with  system  controls  and 

tasks,  such  as  becoming  familiar  with  system  locations,  practice  operational  procedures  and 

controls  and  locations,  practicing  operational  perform  system  malfunction  training.  All  of  this 

procedures,  and  performing  system  malfunction  training  can  be  performed  In  a  simulated 

procedures.  As  conceived  mid-range  trainers  real-time  operational  environment.  The  mid-range 

provide  maximum  "hands-on"  training  In  a  trainer  would  represent  the  highest  level  of 

well-ordered  training  program.  In  many  cases,  simulated  training.  It  must  be  capable  of 

these  mid-range  training  devices  will  be  the  complete  lower-level  training  because  the  next 

"backbone"  of  a  well  ordered,  highly  developed  level  would  be  training  in  the  weapon  system 

training  program.  itself.  The  mid-range  trainer  to  be  cost 

effective,  must  take  the  load  of  training  lower 
TRAINING  SITUATIONS  level  objectives  away  from  the  weapon  system. 

Mid-range  Training  devices  can  be  Another  case  for  use  of  interim  trainers 

effectively  developed  and  applied  in  two  training  would  be  during  major  weapon  system  modification, 

situations:  1)  the  fluid  training  situation,  and  When  a  weapon  system  undergoes  a  major 

2)  the  static  or  established  training  situation.  modification,  the  weapon  system  Is  usually 

modified  before  the  companion  full-range 
In  a  fluid  training  situation,  the  training  simulator  is  either  brought  up  to  date  or 

program  Is  being  developed  and  produced  while  the  produced.  An  Interim  device  will  be  necessary  to 

weapon  system  Is  in  full  scale  development.  Thus  train  new  procedures  and  new  equipment  before  the 

the  word  fluid  Is  applied  because  the  curriculum 
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weapon  system  Is  used.  This  mid-range  trainer 
would  be  used  In  conjunction  with  a  highly 
developed  conversion  training  program.  A 
simulator  attached  to  this  type  of  training 
program  would  be  a  Part  Task  Trainer  or  Cockpit 
Procedures  Trainer,  a  device  specifically 
designed  to  train  only  the  changes,  modifications 
and  specific  tasks  and  functions  related  to  the 
operation  of  the  new  system. 

In  the  static  training  situation,  the 
training  proyram  has  both  mid-range  and 
full-range  trainers.  The  weapon  system  Is  static 
In  that  It  has  been  delivered  and  all  training 
revolves  around  an  established  program  and 
continuous  proficiency  training  situations.  An 
example  of  the  static  training  situation  would  be 
the  Combat  Crew  Training  Program.  In  this 
situation  the  mid-range  trainer  would  be  used  to 
train  at  lower  levels,  preparing  the  trainee  to 
move  to  the  full-range  device.  This  one  fact  Is 
Important  to  the  well  ordered,  established 
training  program  to  provide  cost  effective  use  of 
the  full-range  device.  In  the  simplest  terms, 
the  mid-range  trainer  provides  cost  effectiveness 
by  allowing  the  training  manager  to  use  the 
full-range  device  for  what  It  was  designed  for; 
to  train  higher  level  objectives.  Mid-range 
trainers  could  be  specifically  designed  to  fit 
Into  a  highly  developed  Combat  Crew  Training 
Program.  It  Is  conceivable  that  this  type  of 
device  would  be  used  as  a  "lead-in"  for  training 
missions  In  the  full  range  trainer.  If  the 
trainee  Is  knowledgeable  in  switch  locations 
system  function  and  operation,  the  full  range 
trainer  sessions  can  be  used  for  real  mission 
profiles.  If  problems  are  encountered  In  student 
performance,  that  student  can  be  given  extra 
training  sessions  in  the  mid  range  device  until 
proficiency  has  been  attained.  The  WST  training 
periods  now  become  cost  effective  and  can 
complement  training  In  the  weapon  system  Itself. 

By  moving  through  the  hlerachy  for  training 
devices,  students  become  better  trained  and  build 
the  confidence  that  Is  necessary  to  perform  In 
the  weapon  system. 

The  key  to  any  mid-range  trainer  Is  their 
low  cost,  high  training  value  and  short 
development  time.  Use  of  minicomputers, 
off-the-shelf  components  and  a  modular  design  Is 
Important  to  reduce  the  Initial  cost  of  mid-range 
trainers  and  provide  the  flexibility  necessary 
for  future  modifications  and  expansion  of  these 
devices. 

As  the  next  sections  Illustrate,  the  B-52 
Offensive  Avionics  System  (0AS)/Cru1se  Missile 
Carrier  (CMC)  Part  Task  Trainer  (PTT)  fills  all 
the  necessary  requirements  of  the  concept  of 
mid-range  trainers  as  part  of  a  sophisticated 
training  program. 

INTRODUCTION  TO  THE  B-52  OAS/CMC  PART  TASK  TRAINER 

The  U.S.  Air  Force  B-52  Bomber  is  receiving 
a  major  modification  In  the  form  of  the  Offensive 
Avionics  System  (OAS).  The  OAS,  which  Is  used  by 
the  B-52  radar  navigator  and  navigator  to  perform 
navigational  and  offensive-weapon-delivery  tasks, 
replaces  the  older  ASQ-38  equipment  and  automates 
some  previously  manual  functions. 


As  part  of  a  major  weapon  system 

modification,  a  total  aircraft  slmulator/tralner 
will  be  built  for  the  OAS.  This  full-range 

trainer  Is  scheduled  to  be  operational  In  the 
1983-1986  time  frame,  which  leaves  a  gap  between 
scheduled  deployment  of  the  OAS  and  availability 
of  the  complete  trainer  system.  Crews  using  the 
first  B-52  OAS  modified  aircraft  will  need  a 
trainer  In  the  Interim  to  fill  this  gap.  The 
4235  Strategic  Training  Squadron  (STS), 
Headquarters  Strategic  Air  Command  (SAC),  In 
conjunction  with  Air  Training  Command  and 
Technology  Service  Corporation  of  Santa  Monica, 
CA,  has  been  tasked  to  develop  and  build  four 
B-52  OAS  Interim  trainers  to  be  used  by  Strategic 
Air  Command  In  their  OAS  conversion  training 
program. 

The  4235  Strategic  Training  Squadron  (STS) 
was  tasked  to  develop  the  OAS  conversion 
curriculum.  Included  with  this  tasking,  was  the 
development  of  a  new  OAS/Crulse  Missile  training 
program  and  curriculum  for  the  B-52  Combat  Crew 
Training  School  at  Castle  AFB,  CA.  Using  the 
Instructional  Systems  Development  (ISO)  approach, 
the  4235  STS/Speclal  Project  Branch  developed  the 
training  requirements  for  the  OAS  and  Cruise 
Missile.  With  these  training  requirement  In 
hand,  a  list  of  tasks  outlined  which  were 
essential  for  training  In  a  ground  simulator  was 
developed  (see  Figure  1).  These  tasks  were 
determined  with  both  the  conversion  program 
(fluid  training  situation)  and  the  Combat  Crew 
Training  Program  (static  training  situation)  In 
mind.  As  requirements  and  design  began  to  take 
shape  It  became  readily  apparent  that  an  Intenn 
mid-range  device  was  absolutely  necessary  to  the 
Conversion  Program  and  would  greatly  aid  In 
Combat  Crew  Training. 

The  Interim  trainer  will  be  a  Part  Task 
Trainer  (PTT),  one  that  addresses  some  of  the 
tasks,  procedures  and  conditions  the  crewmembers 
must  handle  with  the  actual  weapon  system  (See 
Figure  2).  Because  the  air  crews  receiving 
training  will  be  highly  experienced  In  the  B-52 
navigational  functions,  the  focus  of  the  PTT  will 
be  on  procedures  training;  specifically,  those 
procedures  unique  to  the  new  OAS.  As  an  example, 
the  PTT  must  respond  exactly  as  the  OAS  would  to 
all  commands  (button  pushes,  switch  activations, 
etc.),  but  only  those  features  used  directly  In 
navigational  and  weapon- targeting  procedures  need 
be  displayed  In  radar  video. 

With  only  the  limited  OAS  functions 
Included,  the  PTT  would  reduce  the  Conversion 
Training  Program  by  two  flights  per  crew,  saving 
the  Air  Force  $16,000,000.  If  applied  In  the 
Combat  Crew  Training  Program  the  device  would 
reduce  WST  periods  necessary  to  train  lower  tasks 
and  again  savings  would  amount  to  approximately 
$400,000  a  year  In  simulator  training  time.  The 
above  savings  alone  are  significant,  but  when 
compared  with  the  cost  of  the  four  simulators 
they  becomr  ipressive.  The  entire  simulator 
project  was  given  $1.9  million,  which  breaks  down 
to  $450,000  per  device  including  software  and 
hardware. 

Design  features  of  the  PTT  hardware  include 
mobility,  so  that  the  PTT  can  be  moved  easily 
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TASKS  FOR  THE  OAS  PART  TASK  TRAINER 


GENERAL  NAVIGATION  TASKS 

Navigation  Tasks 

(To  perform  genaral  OAS 

operation) 


Destination  Sequenclng/Selectlon 
Flxpolnt  Sequenclng/Selectlon 
Direct  Steering 
Centerline  Recovery  Steering 


Position  Fixing  Tasks 

(To  update  aircraft  position  and  course) 


Radar  Position  Fix 
Terrain  Correlation  Fix 
Overfly  Fix 


Calibration  Tasks 


systems) 


High-Altitude  Calibration 
Memory  Point  Wind  Calibration 
Alternate  Heading  Calibration 
Low-Altitude  Calibration 


Rendezvous  Tasks 


refueling) 


WEAPONS  TASKS 


To  prepare  and  launch/ 


release  weapons) 


SRAM  Alignment 
ALCM  Alignment 
SRAM  Prearm 
ALCM  Targeting 
ALCM  Retargeting 
ALCM  Prearm 
SRAM  Ranging 
SRAM  Launching 
ALCM  Ranging 
ALCM  Launching 
Procedure  Irregularity 
Recovery 

Missile  Jettisons 
Launcher  Rotation 
Synchronous  Bomb  Run 
Alternate  Bomb  Run 


MALFUNCTIONS 


To  perform al ternate/back 


up  procedures  when  OAS 
components  fall) 


Point  Parallel  Rendezvous 
Alternate  Air  Rendezvous 


Special  Procedures 


OAS  Turn  On 
OAS  Shutdown 


RN  Management  Panel 
Falls 

WCP  Panel  Falls 
Doppler  Radar  Failure 
OAS  Failure 


Figure  1 
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B-52G/H  OAS/CMC  PART  TASK  TRAINER 


from  base  to  base  to  meet  training  schedule;  and  The  OAS  PT I  automatically  navigates  the  11-bP 

use  of  off-the-shelf  computer  components.  To  according  to  a  predefined  scenario  flight  plan, 

reduced  the  cost  of  hardware  and  make  maintenance  The  crewmembers  are  able  to  monitor,  update,  and 

easy  by  service  contract  decreasing  PTT  down  override  this  automatic  navigation  system,  using 

time.  As  to  the  software,  H  must  be  flexible  to  the  OAS  equipment  and  a  variety  of  OAS 

handle  OAS  design  changes  easily  and  make  future  navigational  procedures  supported  by  the  PTT. 

expansion  possible.  The  PTT  also  supports  OAS  procedures  to  perform 

In-flight  refueling;  preparation  and  delivery  of 
OAS  PART-TASK  TRAINER  CAPABILITIES  Air  Launched  Cruise  Missiles,  Short  Range  Attack 

Missiles,  and  Gravity  Weapon  Delivery;  and  backup 
The  physical  layout  of  the  PTT  (see  Figure  procedures  used  when  certain  OAS  failures  occur. 

3)  comprises  stations  for  the  two  crewmembers  Takeoffs  and  landings  are  not  supported  by  the 

(called  the  radar  navigator  and  the  navigator)  PTT,  but  OAS  startup  and  shutdown  procedures  are 

required  to  operate  the  OAS,  as  well  as  for  an  functional, 

instructor.  Included  in  the  PTT  is  a  physical 

mockup  of  the  OAS  Identical  to  the  OAS's  layout  In  the  PTT,  the  instructor  can  select  a 

and  dimensions  of  panels,  switches,  displays,  flight  plan  from  a  set  of  seven  canned  scenarios 

etc.  The  mockup  comprises  lb  operations  panels,  or  modify  any  canned  scenario  to  generate  a  new 

two  of  which  include  a  keyboard  and  trackball  flight  plan.  The  canned  scenarios  are 

(one  for  each  crewmember).  Four  monochromatic  representative  of  actual  training  missions, 

display  monitors  provide  up  to  33  different  Scenarios  of  up  to  five  hours  of  flight  time  can 

display  formats,  including  three  with  synthetic  be  generated  within  a  region  of  2-mi  1 1  ion  square 

radar  imagery.  Tne  instructor's  position  is  nautical  miles,  covering  the  Western  and  Central 

equipped  with  a  CRT  console  for  setup  and  United  States.  Sufficient  navigational  fixpoints 

monitoring  of  the  training  sessions.  and  target  areas  are  included  within  this  region 

to  define  at  least  1UU  different  realistic 
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scenarios.  Actual  terrain  data  are  used  to 
generate  synthetic  radar  Imagery  for  display  of 
these  navigational  flxpolnts  and  target  areas. 
Since  radar  scope  Identification  Is  not  a  primary 
training  requirement  the  synthetic  radar  will 
contain  only  chosen  fix  points,  target  OAP's  and 
radar  returns  on  flight  path  for  navigation 
orientation. 

The  Instructor  also  has  the  capability  to 
Inject  faults  and  malfunctions  at  specific  times 
In  the  scenario,  as  well  as  at  any  time  during 
the  training  session.  With  the  ability  to  steer 
the  aircraft  off  the  flight  plan  and  to  accept 
corrections  from  the  OAS  crewmembers  for 
recovery,  the  Instructor  acts  as  the  B-52  pilot. 
He  also  monitors  crewmembers'  actions  and  Is  able 
to  freeze  the  training  session  at  any  time,  to 
give  additional  instruction,  and  then  resume  the 
session. 

OAS  PART  TASK  TRAINER  DESIGN 

The  PTT  has  been  designed  In  modules,  to 
facilitate  modifications  as  the  OAS  Itself 
changes  and  as  additional  OAS  trainer 
capabilities  are  Identified.  The  five  modules  — 


Independent  subsystems  —  of  the  hardware 
configuration  are  depicted  In  Figure  4  and 
described  below,  lney  Illustrate  the  use  of  off 
the  shelf  equipment. 

Mainframe  Processing  Subsystem 

The  Mainframe  Processing  Subsystem  (MPS) 
consists  of  a  minicomputer  with  disk  drive, 
magnetic  tape  drive,  and  system  console  (See 
Figure  5).  The  disk  Is  used  for  real-time 
storage  and  retrieval  of  radar  Imagery  data;  the 
magnetic  tape  Is  used  for  transfer  of  software 
and  updates  of  terrain  data. 

The  MPS  is  the  central  processor  for  the 
PTT.  It  Is  responsible  for  logical  and  numerical 
processing  plus  control  of  all  other  subsystems. 
The  PTT  Software  runs  on  the  MPS, 

The  virtual  console  Is  a  CRT  required  for 
the  PRIMUS  Operating  System.  The  disk  drive  is 
required  for  the  operating  system,  for  software 
and  data  file  storage,  and  for  real-time  access 
to  display  files  for  synthetic  radar  imagery 
updates.  The  magnetic  tape  drive  Is  required  for 
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loading  terrain  files  and  software  updates  on  the 
operational  systems  In  the  field.  All 
peripherals  are  standard  PRIME  equipment. 

All  peripheral  devices  on  the  PRIME  550  CPU 
{disk  drive,  magnetic  tape  drive,  and  virtual 
console)  have  standard  Interfaces  supported  by 
PRIME  hardware  and  software. 

The  mainframe  processing  subsystem 
capabilities  are  listed  below  for  the  PRIME  550 
CPU. 

PRIME  550  CPU 

32  Bit  CPU  Architecture 

128  Registers 

512  K  Byte  Error  Correcting  Code  Main 
Memory  (expandable  to  2  M  Byte) 

1  K  Word  Cache 

Single/Double  Precision  Floating  Point 
Arithmetic  In  Firmware 

Up  to  32  DMA  Channels 

16  Asynchronous  Channels 


Up  to  63  Simultaneous  Users 
PERIPHERALS 

PRIME  Virtual  Console  (PT-25  CRT 

Terminal ) 

PRIME  96  M  Byte  Cartridge  Module  Device 
Disk  Drive 

PRIME  9-Track,  800  BPI,  45  IPS  Tape 

Drive 

The  MPS  Interfaces  to  the  Display  Subsystem  (os ) , 
Instructor's  Console  Subsystem  (ICS),  and  Switch 
Control  Unit  (SCU).  These  Interfaces  are 

described  In  the  respective  subsystem 
descriptions  below. 

Display  Subsystem 

The  Display  Subsystem  (US)  is  tne  primary 
output  device  for  the  navigator  and  radar 
navigator  positions.  The  DS  Includes  four 
monochromatic  monitors  (called  multifunction 

displays  or  MFDs)  mounted  on  the  panels  In  the 
crewstatlon.  The  display  processor  and  refresh 
memory  Interface  with  the  mainframe  processor  to 
generate  Imagery,  graphics  and  alphanumerlcs  on 
the  MFDs.  Two  trackballs  are  also  included  In 
the  DS  and  are  physically  mounted  In  the 
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DISPLAY  SUBSYSTEM  BLOCK  DIAGRAM 
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Figure  6 


crews tatlon  to  allow  the  crewmembers  to  position 
crosshairs  on  the  MFUs  (See  Figure  6). 

Each  crewmember  has  control  of  two  MFUs  and 
one  trackball.  Thirty-three  different  display 
formats  may  be  selected  for  presentation  on  any 
one  or  combination  of  MFus.  Most  display  formats 
contain  only  alphanumeric  data,  but  some  contain 
alphanumeric  data  overlaid  on  radar  video 
imagery,  which  at  any  one  time  may  include 
crosshairs,  ground  track  vector,  and  navigational 
markers  (aircraft  heading  vector,  range  rings, 
bezel  rings). 

The  display  processor,  refresh  memory,  and 
trackballs  are  a  special-purpose  system  provided 
by  Grlnnell  Systems  Corporation  for  this 
application.  The  monitors  are  model  NDC-1Z 
provided  by  TSO  Display  Products,  Inc. 

The  US  interfaces  with  the  mainframe 
processor  via  two  high-speed,  unldirectinal 
16-bit  parallel  interfaces  with 
direct-memory-access  (DMA)  transfer  capability. 
To  accomplish  16-bit  parallel  data  transfers,  the 
General  Purpose  Interface  Board  provided  with  the 
PRIME  computer  was  modified  by  Grinnell  to 
interface  with  the  display  controller. 


The  following  is  a  list  of  the  Display 
Subsystem  Capabilities; 


IMAGE 


One  Image  Shared  By  Up  To  Four  Monitors 

4  Bits  (16  Levels)  Gray  Scale  Image 

4  Bits  For  Four  Overlays  (also  shared  by 
moni  tors) 

1024  x  1024  Resolution  (512  x  512 
displayable  at  any  one  time) 


ALPHANUMERICS 


Independent  Alphanumerlcs  On  Each  Monitor 

May  Overlay  Portions  Of  Image  Data 

Dual  Intensity 

Reverse  Video 

Underl  ine 

Blink 

7x9  pixel  characters  In  8  x  12  matrix 
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OAS  CONSOLE  SUBSYSTEM  CONFIGURATION 


Figure  7 


128  alphanumeric  characters,  special  symbols 
MONITORS 

4  Monochromatic  (P39  Phosphor)  Monitors 
6"  x  9"  Viewing  Area  (or  12"  diagonal) 

OAS  Crewstation  Subsystem 


The  SCU  block  diagram  is  shown  in  Figure  7, 
The  SCU  is  a  CROMEMCO  Z2  microcomputer  with 
appropriate  software. 

The  interface  box  (see  Figure  8),  which  is 
mated  with  the  Cromenco  11  microcomputer  to  form 
the  Switch  Control  Unit,  is  being  produced  at  the 
3300  Technical  Training  Wing/Technical  Training 
Division,  Keesler  AFB,  MS, 


Tne  OAS  Crewstation  Subsystem  (see  Figure  7)  Instructor  Console  Subsystem 

contains  operational  mockups  of  those  OAS  and 

weapon  panels  required  for  the  specified  The  Instructor's  Console  Subsystem  (ICS) 

procedures  training  in  the  PTT.  This  subsystem  serves  as  a  system  monitor  and  control  Interface 

consists  of  the  16  panels  used  by  the  for  the  instructor.  It  provides  for  Interchange 

crewmembers.  The  panels  are  equipped  with  of  textual  data  with  the  Part  Task  Trainer 

switches,  buttons,  lamps,  and  two  keyboards.  The  control  programs  executing  in  the  MPS.  The 

four  MFOs  and  two  trackballs  are  physically  following  data  are  displayed  at  the  Instructor's 

located  with  the  panels  in  the  crewstation  but  Console  on  an  alphanumeric  CRT  display: 

are  logically  part  of  the  Display  Subsystem. 

Flight  Parameters 

Switch  Control  Unit 

Mission  Data 

The  Switch  Control  Unit  (SCU)  provides  an 

Interface  between  the  mainframe  processor  and  the  Status  of  OAS-Emulated  Subsystems 

switches  and  lamps  in  the  panels  In  the 

crewstation.  The  SCU  is  responsible  for:  1)  Operational  Faults 

sensing  switch  activations  on  the  panels  and 

reporting  them  to  the  MPS,  and  2)  generating  Alphanumeric  data  and  control  codes  are 

appropriate  signals  to  light/ extinguish  lamps  on  input  through  a  keyboard.  Functions  which  can  be 

the  panels  by  request  of  the  MPS.  exercised  are: 
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Trainer  Session  Control 

Alteration  of  Flight  Parameters 

Alteration  of  Mission  Data 

Modification  of  OAS-Emulated  Subsystems 
Status 

Fault  Seeding 

Recovery  from  Operational  Faults. 

The  ICS  is  a  SOROC  IQ  140  CRT  terminal. 

Tne  instructor  can  interact  with  the  PTT  In 
three  operational  modes:  Pre-Run,  Run,  and 

Post-Run.  The  interface  has  been  designed  to 
facilitate  moving  back  and  forth  among  any  of  the 
three  modes  (see  Figure  9). 

The  Pre-Run  mode  occurs  prior  to  the 
training  (simulation)  session  and  allows  the 
instructor  to  select  and  edit  a  scenario  for 
training,  and  complete  setup  and  initialization 
procedures.  The  Run  mode  Is  the  actual  training 
period  or  simulation,  during  which  the  Instructor 
monitors  the  crevnembers  actions  and  Injects 
changes  Into  the  session.  The  Post-Run  mode 
occurs  after  the  training  session.  In  which  the 
Instructor  can  Initiate  a  limited  review  and 
analysis. 

All  sessions  must  go  through  some  minimum 
initialization  via  the  Pre-Run  mode  of  the  user 
Interface.  After  the  appropriate  Initialization, 
the  Instructor  will  enter  the  Run  mode,  via 


commands  available  to  him,  to  begin  the  training 
session.  He  will  be  provided  with  commands  that 
allow  him  to  abort  the  session  and  return  to  the 
Pre-Run  mode,  or  Interrupt  for  a  time  period  and 
then  resume  within  the  Run  mode  Itself.  Once  the 
run  Is  complete,  the  Instructor  may  enter  the 
Post-Run  mode,  In  which  he  may  conduct  limited 
review  and  analysis,  or  he  may  return  to  the 
Pre-Run  mode  to  Initiate  another  training 
session. 

The  software  has  a  modular  design,  which 
will  aid  In  making  future  changes.  Software 
development  Is  performed  on  the  mainframe 
procesor  In  a  high-level  language.  The  PTT 
software  package  with  the  exception  of  the  Switch 
Control  Unit  software  is  being  designed  and 
produced  by  Technology  Service  Corporation  of 
Santa  Monica,  CA.  The  Switch  Control  Unit 
software  Is  being  developed  by  the  3300  Technical 
Training  Wing/Technical  Training  Division  at 
Keesler  Air  Force  Base,  MS.  The  entire  software 
package  outlined  in  the  above  paragraphs  will  be 
an  18  month  effort.  Funding  for  the  B-52  OAS/CMC 
PTT  was  received  on  27  April  80  and  first 
production  unit  will  be  delivered  to  Strategic 
Air  Command  on  15  Oct  81. 

SUMMARY 

A  full-capability  trainer  cannot  be 
delivered  prior  to  OAS  deployment,  yet  procedures 
training  Is  essential  for  the  OAS  conversion 
training  and  operational  use  of  the  new  system. 
The  B-52  Offensive  Avionics  System  Part  Task 
Trainer  is  a  powerful  training  system  provided  at 
a  low  cost  and  In  a  short-term  schedule. 
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Undoubtedly  there  are  other  military  systems 
whose  training  requirements  can  also  be  met  by  a 
mid-range  trainer.  Such  systems  are  likely  to 
require  software  control  of  switch  panels, 
real-time  Instructor  Interaction,  multiple 
displays,  and  graphics  displays— requirements 
which  are  being  satisfied  by  the  PTT.  The 
modularity  and  flexibility  being  designed  Into 
the  hardware  and  software  of  the  PTT  will 
minimize  the  efforts  required  to  adapt  this 
design  to  other  applications. 

REFERENCE 

1.  Offensive  Avionics  System  Part  Task  Trainer, 
Software  Performance  Specification, 
TSC-PQ-A248-6,  December  8,  1980,  Revised  1  March 
1981. 

2.  Offensive  Avionics  System  Part  Task  Trainer 
Specification  Document,  TSC-PD-A248-3,  August  lb, 
1980,  Revised  October  6,  1980. 

3.  Instructor's  Manual,  TSC-PD-A248-3,  October 
7,  WW. 


ABOUT  THE  AUTHORS 

Captain  Joseph  C.  Stein  Is  asslgred  to  the  4235 
Strategic  Training  Squadron,  Headquarters 
Strategic  Air  Command  (SAC).  His  present  duty 
title  Is  OAS/Crulse  Missile  Training  Manager.  He 
Is  the  SAC  Project  Manager  for  the  B-52  OAS/CMC 
Part  Task  Trainer.  Captain  Stein  holds  a  BS 
degree  In  Economics  from  the  University  of 
Southwestern  Louisiana,  Lafayette,  LA,  and  MBA 
degree  In  Management  from  Golden  Gate  University, 
San  Francisco,  CA. 

Captain  Michael  E.  Shannon  Is  assigned  to  the 
4235  Strategic  Training  Squadron,  Headquarters 
Strategic  Air  Command  (SAC).  His  present  duty 
title  Is  OAS/Crulse  Missile  Training  Manager.  He 
Is  working  on  the  development  of  the  B-52  OAS/CMC 
Part  Task  Trainer.  Captain  Shannon  holds  a  BA  In 
Economics  from  the  University  of  Montana, 
Missoula,  MT. 

Ms.  Carole  J.  Kuruma  Is  Training  Systems 
Department  Manager  at  Technology  Service 
Corporation,  Santa  Monica,  CA.  She  Is  Project 
Team  Manager  for  the  B-52  OAS/CMC  Part  Task 
Trainer  Software  Development.  Ms.  Kuruma  holds  a 
BA  degree  In  Mathematics  from  California  State 
University  at  Northrldge  and  a  MS  In  Computer 
Science  from  UCLA,  Los  Angeles,  CA. 


355 


r  in -ftlfiflriWiri  r  •'•T  r — 


FOREIGN  MILITARY  TRAINING  PROGRAM  DEVELOPMENT 
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ABSTRACT 


Modernization  of  the  military  posture  of  friendly  foreign  governments  has  placed 
uncontemplated,  challenging  demands  on  training  program  developers.  To  minimize  total 
cost,  foreign  governments  use  existing  equipment  designs  where  feasible.  By  using  existing 
designs,  the  logistics  support  package  has  already  been  developed;  however,  this  package 
was  tailored  to  meet  the  needs  of  United  States  military  personnel.  All  items  of  the 
loqistics  package  are  satisfactory  except  —  training  and  technical  publications.  Problems 
in  these  two  areas  become  especially  difficult  when  English  is  used  as  a  second  language  by 
the  procuring  country.  Since  their  background  and  experiences  are  different,  a  training 
approach  that  is  different  than  the  one  used  in  the  existing  package  is  required.  Training 
developers  must  modify  existing  curriculum  to  allow  for  these  differences,  to  remove  cultur¬ 
ally  offensive  situations,  and  to  tailor  the  training  to  learning  patterns  of  the  intended 
student.  Once  the  training  program  accomplishes  its  objective,  problems  with  the  technical 
documentation  disappear. 


INTRODUCTION 

Military  technology  is  advancing  rapidly 
requiring  continual  updating  of  even  the  most 
modern  weapon  systems.  Many  of  our  allies  maintain 
a  modern  defensive  posture  through  procurement  of 
sophisticated  weaponry  from  the  United  States  and 
its  No’-th  Atlantic  Treaty  Organization  Allies. 

Existing  equipments  and  support  packages  are 
usually  procured,  where  feasible.  Though  most 
elements  of  the  support  package  are  universally 
acceptable ,  two  of  these  can  prove  troublesome . 

They  are;  training  and  technical  manual  programs. 
These  may  prove  troublesome  because,  of  all 
elements  of  the  support  package,  these  two  are 
developed  to  fulfill  the  specific  needs  of  a 
group  of  military  weapon  systems  support  techni¬ 
cians  and  operating  specialists  who  share  a  common 
core  of  training  and  experience.  Unless  .the 
support  technicians  and  operating  specialists  of 
the  procuring  country  share  this  common  training 
and  experience,  attempts  to  use  the  training 
portion  of  the  support  package  can  be  expected  to 
result  in  less  effective  training. 

A  recent  example  where  the  use  of  existing 
training  programs  to  save  time  and  money  proved 
unsatisfactory  was  when  students  of  the  middle 
east  were  placed  in  nine  different  training  pro¬ 
grams  that  had  been  developed  for  U.S.  Navy 
technicians.  Existing  programs  were  used  in  an 
attempt  to  save  time  and  money.  Each  program  was 
given  to  four  different  groups  over  an  extended 
period.  To  insure  that  problems  would  be 
identified  immediately,  frequent  evaluations  of 
training  progress  were  planned.  Because  of 
language,  background,  training,  and  experience, 
differences  which  existed  between  the  U.S.  studen  s 
the  group  for  which  the  programs  had  been  developed, 
many  problems  were  identified. 


programs  for  use  to  train  personnel  who  possessed 
different  qualifications  than  those  for  which  the 
training  had  been  developed. 

As  each  problem  was  identified,  the  training 
program  was  revised  in  an  attempt  to  eliminate 
like  problems  for  those  groups  programmed  to 
follow. 

The  end  result  was  that  each  group  success¬ 
fully  completed  their  respective  training  pro¬ 
grams.  This  was  made  possible  because  of  the 
extreme  number  of  hours  that  had  been  devoted  to 
program  revision  and  increased  training  time. 
However,  students  would  have  suffered  less  frus¬ 
tration  and  total  cost  would  have  been  le.s  if  the 
training  programs  had  been  initially  developed  to 
satisfy  the  specific  needs  of  this  type  of  student. 

THE  INITIAL  ATTEMPT 
Program  Development 

In  the  initial  attempt,  training  courses  were 
established  around  existing  U.S.  military  curricu¬ 
la  in  every  respect,  including  course  length, 
course  content,  course  subject  sequencing,  labora¬ 
tory  and  classroom  time,  and  selected  media.  As 
mentioned  above ,  this  consisted  of  nine  courses 
of  instruction  which  was  repeated  to  different 
classes . 

The  Instructor 

In  each  case,  experienced,  highly  competent 
course  developers  and  instructors  were  used. 
Instructor  personnel  had  been  specifically 
selected  and  their  performance  observed  and  eval¬ 
uated  prior  to  the  beginning  of  training.  Those 
personnel  allowed  to  teach  had  been  certified  by 
the  contractor  and  by  the  Government. 


As  it  turned  out,  the  first  group  of  students  First  Indication  of  Trouble 

served  a  dual  role.  In  addition  to  becoming 

qualified  in  their  selected  specialty,  they  became  At  the  end  of  the  first  week,  in  each  of  the 

a  pilot  group  for  validation  of  existing  training  nine  courses,  the  instructors  were  behind  the 

planned  schedule  by  30%  to  50%. 
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Student  Reports 


When  questioned  about  their  training,  sti  ' int 
comments  for  all  courses  were  similar,  and  con¬ 
sisted  primarily  of  the  fol lowing i 

1  -  "Course  is  too  fast" 

2  -  "instructor  is  no  good" 

3  -  "Instructor  doesn't  explain  well" 

4  -  "Course  is  too  difficult" 

5  -  "We  don't  have  enough  time" 

6  -  "Don't  need  to  take  teets" 

7  -  "Wo  need  to  see  and  use  the  equipment" 

ANOTHER  ATTEMPT 
Override  in  U.S.  Courses 

In  addition  to  the  above  nine  courses,  many 
personnel  with  similar  cultural  backgrounds  were 
provided  English  Language  Training  (ELT)  then 
entered  in  U.S.  military  training  courses,  to 
compete  directly  with  U.S.  military  personnel. 

Rer.ult 

The  resulting  attrition  rate  of  these 
personnel  was  intolerable,  especially  for  those 
students  enrolled  in  electrical  and  electronic 
type  courses.  This  attrition  rate,  in  many  cases, 
exceeded  50%. 

Possible  Problem 

Though  all  courses  were  written  and  pre¬ 
sented  in  English,  this  was  not  considered  a 
problem  because  the  students  were  first  cycled 
through  English  Language  Training,  at  Lackland 
Air  Force  Base,  San  Antonio,  Texas.  A  requirement 
for  graduation  from  this  course  was  that  they 
demonstrate  attainment  of  an  English  Comprehension 
Level  (ECL)  of  70.  This  ECL  of  70  was  initially 
equated  to  the  7th  grade;  however,  later  investi¬ 
gation  revealed  that  this  score  was  based  on  a 
test  that  primarily  evaluated  their  use  and 
recognition  of  spoken  English.  Little  reading 
ability  or  reading  comprehension  was  checked. 

Research  into  the  Problem 

Because  of  this  high  attrition  rate,  the 
Government  investigated  the  situation  and  con¬ 
cluded  that  the  problem  was  a  multifaceted  one. 

The  researchers  surmised  that  background  experi¬ 
ences,  cultural  trait3,  and  different  patterns  of 
learning  were  all  factors  which  contributed  to 
their  undesirable  performance  in  the  learning 
situation. 

Learning  Pattern  Differences 

Researchers  discovered  that  these  Mid-Eastern 
cultures  learn  predominantly  through  rote  memory, 
whereas  U.S.  curricula  was  based  on  the  premise 
that  the  student  should  first  learn  principles, 
then  he  could  be  led  through  attainment  of  the 
training  objective  through  analysis  of  the  problem 
and  a  synthesis  of  basic  principles  to  provide 
solutions  to  the  problem.  Consequently,  an 
anomaly  in  training  strategy  had  existed  from  the 
beginning. 


Evaluation  of  Learning 

U.S.  military  curricula  is  based  upon  a 
pattern  of  continued  feedback.  To  insure  that  the 
student  is  learning  sufficiently  to  enable  him  to 
accomplish  the  objective,  the  instructor  is 
directed  to  provide  interim  summaries  and  evalua¬ 
tions  at  strategic  points  within  the  lesson.  In 
other  words,  the  instructor  directs  thought-provok¬ 
ing  questions  at  specific  students  in  order  to 
determine  whether  that  student  has  learned  the 
subject  to  that  point,  and  to  insure  that  he  does 
not  have  misunderstandings  about  key  points  of  the 
lesson.  In  order  to  insure  an  adequate  sampling 
of  student  personnel  for  obtaining  this  feedback, 
the  instructor  is  trained  to  randomly  select  and 
direct  the  question  to  an  individual  student. 
Because  of  this  procedure,  a  student  that  is  not 
understanding  the  subject  will  not  evade  detection 
by  the  instructor  for  very  long.  This  system  is 
used  regardless  of  the  rank  or  3tatus  of  individ¬ 
ual  members  of  the  class.  This  works  fine  for 
U.S.  students  because  they  are  considered  to  be  of 
equal  rank  during  that  time  when  they  are  in  the 
learning  situation.  No  one  student  has  a  higher 
status  than  other  students  of  the  class. 

This  same  philosophy  with  Mid-Eastern 
students  met  with  sudden,  overwhelming  resistance. 
First,  their  cultural  training  seems  to  prevent 
them  from  engaging  in  any  act  or  activity  that  may 
prove  degrading  to  another  member  of  their  society. 
Consequently,  if  one  student  does  not  know  the 
answer  to  a  question,  it  would  be  degrading  to  him 
for  another  studant  to  provide  the  correct  answer. 
Initial  reaction  of  the  student,  when  placed  in 
this  situation,  seems  to  be  that  if  he  doesn't 
provide  an  answer  he  cannot  be  wrong.  When  ques¬ 
tioned  about  quizzes  and  exams,  especially  of  the 
oral  type,  their  reaction  was  that  they  were  there 
to  learn,  not  to  be  tested.  It  is  interesting  to 
note  that  this  attitude  is  :  upported  by  the  con¬ 
cept  of  rote  memory  as  opposed  to  the  analysis  and 
synthesis  process  mentioned  earlier.  In  other 
words,  if  they  are  given  the  question  and  the 
correct  answer,  they  can  memorize  it.  This  is 
their  customary  means  of  learning.  If  they  learn 
through  this  method  of  instruction,  there  is  no 
need  for  tests  or  quizzes,  and  no  competitiveness 
occurs.  Therefore,  no  student  is  caused  to  seem¬ 
ingly  degrade  another  student,  especially  one  who 
has  the  higher  rank  or  status.  Usually  when  a 
question  is  posed  to  a  class  of  students  that  has 
mixed  rank,  the  duty  of  providing  the  answer  falls 
upon  the  ranking  member.  All  subordinate  members 
of  the  class  then  support  that  answer  given  by 
their  leader. 

When  confronted  with  a  written  exam,  whereby 
each  individual  must  provide  an  answer  of  his  own, 
the  students  often  start  talking  among  themselves 
in  their  native  tongue.  A  natural  reaction  of  the 
instructor  is  to  believe  that  they  are  cheating. 
However,  when  the  instructor  intervenes,  the 
ranking  class  member  usually  informs  him  that  they 
were  discussing  the  time,  or  some  other  unrelated 
thing.  They  seem  to  feel  that  their  group  has  been 
confronted  with  a  problem  rather  than  each  individ¬ 
ual  having  been  confronted  with  the  problem.  This 
justifies  their  free  discussion. 
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Dependence  upon  Instructor 


Through  observation  and  questioning  of  these 
students,  their  successful  instructors,  and  train¬ 
ing  administrator  personnel,  it  was  determined 
that  the  students  come  to  see  their  instructor  as 
a  friend  and  confidant.  They  develop  a  dependence 
upon  their  instructor  for  everything,  not  just 
training  related  problems.  They  bring  any  problem 
to  him,  and  they  trust  and  rely  heavily  upon  his 
judgment.  He  is  their  friend,  their  counselor, 
and  their  advisor  as  well  as  being  their 
instructor.  However,  not  every  individual 
instructor  was  able  to  develop  such  rapport  with 
them.  Because  of  this,  an  instructor  that 
possesses  a  great  deal  of  experience,  who  may  even 
be  considered  a  leader  in  the  field,  and  who  has 
been  an  excellent  instructor  for  U.S.  military 
personnel  may  not  be  considered  a  good  instructor 
by  such  students.  This  possibly  resulted  because 
the  foreign  students  did  not  feel  safe  in  trusting 
him  with  their  individual  problems. 

Similarity  to  the  Job 

Because  U.S.  military  personnel  could  be 
expected  to  operate  any  one  of  perhaps  100 
different  pieces  of  equipment  that  has  been 
designed  to  provide  a  specific  function,  only  a 
piece  of  equipment  that  is  most  representative  of 
the  group  of  100  different  types  is  selected  for 
use  in  the  training  course.  For  example,  the 
U.S.  Navy  may  have  within  its  inventory  of  active 
equipment  as  many  as  100  different  types  of 
communication  receivers.  A  type  is  selected  for 
use  in  the  training  course  that  i3  deemed  to  be 
most  representative  of  the  other  99  types. 

Though  students  are  trained  on  this  one  type,  they 
are  expected  to  relate  their  training  to  any  type 
with  which  they  may  be  required  to  operate.  They 
are  taught  the  function  of  the  "SQUELCH"  control 
and,  regardless  of  where  the  control  may  be  found 
on  the  panel,  they  can  relate  it  to  the  Squelch 
function.  Consequently,  they  are  considered 
qualified  to  operate  the  "SQUELCH"  control  to 
achieve  its  intended  function  on  any  one  of  the 
100  different  types  of  equipment  that  they  may 
be  assigned. 

Through  conditioning  provided  by  the  “rote 
memory"  type  training  which  they  have  received 
throughout  their  life  time,  Hid-Eastern  students 
do  not  tend  to  learn  the  function  of  the  "SQUELCH1' 
knob,  instead,  they  tend  to  learn  the  function 
of  the  knob  located  in  the  lower  left-hand  corner 
of  the  receiver  that  is  labeled  "SQUELCH". 

Because  of  this,  they  do  not  readily  relate  to  a 
different  receiver  that  has  a  knob  near  the 
center  of  the  panel  labeled  "SQUELCH".  They  have 
not  been  trained  to  operate  this  new  knob.  This 
is,  of  course,  oversimplified,  but  used  to 
illustrate  a  problem  which  has  been  identified 
through  external  evaluation  of  their  training 
programs . 

Consequently,  the  Mid-Eastern  student 
expects  to  have  received  training  on  that 
specific  piece  of  equipment  that  he  is  expected 
to  operate . 


Comprehension 

The  use  of  English  by  the  instructor  did  not 
seriously  impede  the  students'  understanding,  as 
long  as  the  instructor  was  careful  to  insure  that 
the  presentation  was  preceded  by  an  identification 
of  new,  unfamiliar  technical  words  along  with  a 
definition  of  their  meaning. 

When  the  lesson  dealt  with  a  piece  of  equip¬ 
ment,  it  was  best  to  have  the  equipment  or  a  good, 
complete  picture  of  it  readily  available.  Since 
their  primary  mode  of  learning  is  rote  memory,  it 
is  best  to  have  the  actual  equipment  available  so 
they  can  have  the  operation  or  procedure  demon¬ 
strated  by  the  instructor,  then  they  can  perform 
the  operation  or  procedure  themselves,  on  that 
piece  of  equipment. 

If  the  instructor  feels  they  should  take 
notes ,  the  only  way  he  can  get  them  to  do  so  is 
to  write  them  on  the  board.  It  seems  that  they 
write  in  their  notes  everything  that  their 
instructor  writes  on  the  board  for  them. 

The  students  greatest  impediment  to  learning 
was  found  to  be  his  reading  comprehension.  One 
major  problem  was  that  he  was  given  study 
materials  that  had  been  written  at  or  above  the 
twelfth  reading  grade  level.  Since  the  average 
reading  grade  level  of  these  personnel  was  below 
the  sixth  grade,  a  built-in  problem  existed. 
Additional  problems  caused  by  using  this  material 
were  attributed  to  the  lack  of  illustrations.  In 
order  for  written  materials  to  be  effective  for 
personnel  who  learn  primarily  by  rote  memory,  it 
is  necessary  to  frequently  illustrate  the  written 
word  through  use  of  accurate,  detailed  pictures 
or  drawings  of  the  equipment  or  system  being 
described. 

FACTORS  CONSIDERED  IN  CURRICULUM 
REVISION  TO  SATISFY  FOREIGN  TRAINING 

The  Training  Situation 

For  this  type  student,  it  was  determined 
that  the  best  training  situation  existed  when 
the  student  was  instructed  on  the  equipment  in  a 
laboratory  type  situation.  This  capitalizes  upon 
one  of  their  strengths  —  that  of  rote  memoriza¬ 
tion.  They  are  adept  at  learning  by  doing. 

Of  course,  this  is  not  always  possible. 
Sometimes  it  is  necessary  that  principles  be 
learned  which  cannot  be  readily  demonstrated  on 
a  piece  of  equipment.  An  excellent  example  of 
this  is  the  theory  of  operation  of  an  electronic 
circuit.  Try  as  one  may,  he  cannot  see  the 
electrons  moving  through  the  wires.  It  is 
necessary  to  resort  to  pictures,  illustrations, 
movies,  and  chalkboard  work  to  supplement  a 
lecture  dealing  with  analogies  between  elec¬ 
tronic  theory  and  known  principles  which  can 
be  readily  viewed,  such  as  water  systems. 

Since  some  instruction  cannot  be  accom¬ 
plished  through  equipment  "hands-on",  it  is 
necessary  to  resort  to  classroom  presentation. 

This  type  training  situation  should  be  preceded 
by  a  demonstration  of  the  job  for  which  this 
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training  is  designed  to  support.  This  allows  each 
student  to  become  involved,  to  a  limited  degree, 
in  the  performance  of  the  jobi  allows  him  to  ask 
questions  regarding  this  operation,  and  answer 
those  which  can  be  answered.  For  those  which 
cannot  be  easily  answered  because  of  the  lack  of 
background  information,  the  instructor  should  show 
here  th^  necessity  of  the  classroom  training,  then 
proceed  to  the  classroom. 

Curricula  developed  to  support  this  training 
should  include  many  pictures  and  illustrations  and 
should  also  make  use  of  the  actual  equipment.  This 
training  should  Always  be  performance  oriented. 

Written  Student  Materials 

For  training  curricula  developed  to 
MIL-STD-1379A,  written  student  materials  include 
Student's  Guide,  Tests,  Equipment  Utilization 
Handbook,  and  the  On-the-Job  Training  Handbook. 

These  materials  should  be  written  to  take 
advantage  of  the  student's  strengths.  Since  he 
has  been  found  to  possess  highly  developed  visual 
memories,  they  can  retain  most  readily  that  which 
they  see.  Consequently,  this  written  material 
should  be  supplemented  heavily  with  pictures  and 
sketches  which  illustrate  what  the  written  word 
is  trying  to  say. 

In  addition  to  illustrative  supplements, 
when  the  written  material  is  trying  to  explain 
something,  the  writer  should  attempt  to  draw 
analogies  between  that  complex  thing  he  is  trying 
to  explain  and  something  that  is  similar  in 
operation  but  less  complex. 

In  selecting  pictures,  illustrations,  and 
analogies,  one  should  insure  that  no  culturally 
offensive  situations  are  created  in  this  material. 
For  example,  pictures  should  not  be  used  where 
individuals  have  the  fingers  of  their  left  hand 
near  their  face.  Also,  analogies  should  not  be 
made  between  a  series  electrical  circuit  and  a 
string  of  Christmas  tree  lights.  These  are  only 
examples,  to  insure  that  materials  are  not  devel¬ 
oped  which  are  culturally  offensive,  developing 
personnel  should  be  made  aware  of  differences 
which  exist  between  their  society  and  ours . 

Editing  personnel  should  also  be  made  aware  of 
these  differences. 

Selection  of  Training  Equipment 

Curricula  written  for  U.S.  military  training 
courses  is  normally  supported  by  equipments  that 
are  generic  to  the  field  of  equipments  upon  which 
the  training  is  designed  to  support.  This  is 
appropriate  because  the  training  approach  is  to 
teach  principles,  then  allow  the  student  to  adapt 
to  specific  needs  through  a  recall  and  synthesis  of 
appropriate  principles.  These  students  learned 
operation  of  a  piece  of  equipment  thoroughly 
through  their  rote  memory  learning  method.  A 
problem  arose  if  they  were  introduced  to  a  version 
of  this  same  equipment  which  functioned  in  an 
identical  manner,  but  which  contained  a  different 
arrangement  for  its  functional  controls.  These 
students  believed  they  should  receive  training 
on  this  new  piece  of  equipment  also.  Consequently, 
the  training  courses  must  be  equipment-specific. 
That  is,  the  equipment  selected  to  support  the 
training  must  be  a  comprehensive  grouping  of  those 


equipments  which  the  graduate  will  be  expected  to 
operate. 

Testing  and  Evaluation 

Because  of  the  desire  among  students  not  to 
compete  with  each  other,  or  disagree  with  any  one 
of  the  others  openly,  it  is  desirable  to  have  them 
complete  individual  tests  on  "easy-to-mark"  answer 
sheets.  That  way,  no  student  will  know  the  answer 
given  by  the  others  in  the  class. 

Planned  Course  Length 

To  enhance  the  rote  learning  profess  and  adapt 
the  presentation  strategy  to  other  unique  student 
strengths,  requires  considerably  longer  course 
items,  especially  for  non-mechanical  subjects. 

Reading  Grade  Level 

Those  curriculum  materials  written  for  student 
use  in  support  of  his  training  should  be  written  to 
a  reading  grade  level  (RGL)  compatible  with  that  of 
the  student  personnel.  For  those  which  have  just 
completed  English  Language  Training  (ELT)  of  the 
type  given  at  Lackland  Air  Force  Base,  where  the 
requirement  for  graduation  is  an  English  Compre¬ 
hension  Level  (ECL)  of  70,  this  falls  somewhere 
between  the  4th  and  6th  reading  grade  level. 

Because  of  this,  materials  written  for  their  use 
during  the  first  six  weeks  of  training,  immediately 
following  ELT,  should  not  exceed  the  6th  reading 
grade  level. 

Because  reading  comprehension  can  be  expected 
to  improve  through  use,  and  because  the  student  is 
rapidly  increasing  his  technical  vocabulary  for 
his  field  of  study,  the  RGL  of  materials  can,  and 
should  be  increased  as  he  progresses  through  his 
rate  training. 

However,  care  should  be  taken  to  insure  that 
no  sample  of  the  student  materials  is  allowed  to 
go  beyond  an  RGL  of  8.9,  as  measured  by  the  Flesch- 
Kincaid  procedure  for  determining  the  Reading  Grade 
Level  of  written  materials. 

Though  most  U.S.  Navy  technical  manuals,  when 
evaluated  using  the  Fry  or  Flesch-Kincaid  methods, 
prove  to  be  written  at  or  near  the  12th  reading 
grade  level,  I  do  not  believe  that  any  attempt 
should  be  made  to  raise  the  reading  grade  level  of 
this  type  student  above  the  9th  grade  because  of 
the  amount  of  time  it  would  take.  Since  their 
basic  military  training,  and  all  specialized  rate 
training  is  being  conducted  in  English,  their 
reading  proficiency  can  be  expected  to  improve 
through  normal  use  as  they  progress  through  their 
specialized  rate  qualification  training.  I 
believe  that  the  motivated  student  who  meets  the 
sixth  reading  grade  level  requirement  upon  entry 
into  his  rate  training  can  be  expected  to  increase 
his  proficiency  to  the  ninth  reading  grade  level 
upon  graduation.  This  leaves  a  three-grade 
discrepancy  between  his  reading  proficiency  and  the 
level  to  which  his  technical  documentation  is 
written.  It  has  been  my  experience  that  this 
discrepancy  does  not  represent  an  insurmountable 
problem.  I  am  convinced,  after  having  studied 
this  problem,  that  an  individual  can  adequately 
comprehend  materials  written  up  to  three  reading 
grade  levels  above  his  own,  if  provided  sufficient 
time. 
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Standardisation 


When  dealing  with  personnel  of  friendly 
foreign  governments .  there  is  much  room  for  mis¬ 
understanding  because  of  differences  in  the  way 
in  which  the  two  countries  conduct  business,  and 
communication  difficulties  which  result  from  the 
different  languages. 

Because  of  this,  it  is  essential  that  all 
deliveries  of  training  material  follow  the  same 
concept  and  format  as  that  with  which  th<  j  are 
familiar.  Since  previously  developed  curriculum 
materials  were  developed  to  MIL-STD-1379A,  and 
its  associated  Data  Item  Descriptions,  it  is 
recommended  that  this  remain  the  standard  which 
governs  the  type  of  curriculum  materials,  their 
content  and  format. 


SUMMARY 

Curricula  developers  and  instructors,  who 
present  the  planned  training  for  foreign  military 
personnel,  must  be  reedy  to  change  thought  patterns 
and  complete  considerable  learning  and  research  if 
they  expect  success.  They  must  temper  their 
thoughts  relative  to  what  constitutes  a  good  train¬ 
ing  program  in  accordance  with  differences  which 
exist  between  their  new  target  population,  versus 
the  population  for  which  they  have  been  writing 
and  presenti.'.j  training. 

Educators:  must  become  thoroughly  knowledgeable 
of  cultural  differences,  i.e.,  different  religious 
beliefs,  different  patterns  of  learning,  concepts 
of  training,  and  background  experiences.  They 
must  design  the  curriculum  around  a  training  concept 
that  takes  advantage  of  the  strengths  of  the  target 
population,  while  overcoming  their  weaknesses. 

Much  research  will  be  required  before  they  will  be 
able  to  identify  these  differences,  strengths  and 
weaknesses,  and  this  research  must  be  completed 
before  they  begin  developing  a  concept  of  the 
training  situation. 
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NEW  FRONTIERS  FOR  COMPUTER  AIDED  TRAINING 
Ing.  Roberto  PodestA 
Gajon  Institute  of  Technology  S.p.A. 

Genoa-Italy 

ABSTRACT 

Computers  have  been  introduced  in  training  since  many  years,  but  with  good  results 
only  for  limited  subjects.  The  main  reason  of  the  failure  seems  to  be  the  passive 
rble  of  the  student  not  sufficiently  motivated  by  the  simple  contact  with  the  ma¬ 
chine.  The  System  we  are  introducing  combines  the  programmed  learning  method  we  de¬ 
veloped  for  military  technicians  through  30  years  of  experiences,  in  which  the  stu¬ 
dent  himself  discovers  the  phenomena  reaching  the  theory  through  experiments,  and 
the  possibility  offered  by  today  Computer  Science,  getting  the  advantages  of  both, 
in  courses  starting  from  basic  subjects  up  to  the  most  advanced  electronic  techni¬ 
ques.  In  a  classroom  equipped  by  our  system  the  individual  computer  gives  the  stu¬ 
dent  a  procedure  for  the  experiments  he  has  to  perform,  but  the  student  himself 
has  to  work  on  an  especially  designed  desk  to  set  the  circuit  and  to  study  it,  mak¬ 
ing  measurements  and  giving  the  computer  his  conclusions.  The  computer  can  evalu¬ 
ate  student's  answers,  letting  him  proceed  if  they  are  correct,  asking  him  to  repeat 
in  the  opposite  case,  adding  other  explanations  to  help  him  and  keeping  his  score. 
Furthermore,  it  controls  an  integrated  audiovisual  system,  showing  to  the  student 
automatically  films  or  transparencies  connected  with  the  current  group  of  experi¬ 
ments  he  is  performing.  Instructor's  function  has  not  been  cancelled,  but  exalted, 
because  through  his  master  computer  he  can  control  the  complete  classroom,  checking 
students'  work  at  any  step,  and  helping  the  one  who  really  needs  his  presence,  not 
disturbing  the  others.  Resuming,  we  realized  a  system  in  which  the  student  has  at 
his  disposal  two  instructors,  the  computer  for  the  routine,  and  the  teacher  when 
necessary  proceeding  at  his  own  pace. 


THE  LIMITS  OF  COMPUTER  AIDED  APPROACH  TO  THE 
TRAINING  OF  MILITARY  TECHNICIANS. 

Talking  about  Computer  Aided  Instruction  could  first  one  is  the  cost  of  such  equipment,  which 

seem  obvious  today,  as  this  kind  of  machines  is  must  be  able  not  only  to  store  this  amount  of  in- 

more  and  more  influencing  our  way. of  life.  formations,  but  also  to  identify  and  correct  a 

As  a  matter  of  fact  the  effort  to  apply  compu-  number  of  possible  mistakes,  which  is  very  big  due 

ters  to  this  important  field  started  since  a  very  to  the  wideness  of  the  subject  itself, 

long  time,  probably  with  Computer  Science  itself,  If,  as  we  told  above,  it  is  relatively  easy  to 

but  if  we  look  deeply  inside  the  results,  we  can  implement  a  program  to  teach  a  technician  how  to 

see  that  these  results  are  encouraging  only  when  use  a  new  component  or  a  new  piece  of  equipment,  it 

limited  subjects  are  involved.  could  be  more  difficult  for  a  computer  to  cancel 

It  is  not  easy  to  resume  in  a  few  words  the  student's  doubts  about  Ohm's  Law,  and  in  no  other 

reasons  of  this  partial  failure  (which  someone  can  field  as  in  computer  business  the  difficulty  of  the 

call  a  partial  success),  but  we  think  that  it  was  problem  is  increasing  the  final  price, 

mainly  caused  by  an  originary  vice,  still  uncancell  But  the  second  point  is  even  more  important.  To 

ed:  the  unlimited  trust  in  the  computer  as  in  a  maintain  and  to  repair  electronic  equipment,  a 

sort  of  magic  box,  which  is  able  by  itself  to  solve  technician  does  not  only  need  theoretical  know- 

any  problem,  without  taking  into  account  that  the  ledge. 

learning  process  is  the  result  of  a  lot  of  elements,  What  he  knows  about  electronics  must  be  inte- 

and  that  it  is  not  realistic  to  pretend  to  repro-  grated  by  his  ability  to  use  it  to  solve  practical 

duce  this  process  simply  by  putting  a  student  in  problems;  it  means  to  be  able  to  use  measuring  in¬ 
front  of  a  machine.  struments,  to  correlate  different  phenomena  and, 

In  this  way  we  can  explain  the  fact  that  good  last  but  not  least,  to  use  his  own  hands, 

results  have  been  achieved  when,  for  example,  what  Moreover,  if  the  technician  we  are  talking 

we  call  "the  student"  was  a  previously  trained  about  is  a  military  one,  the  difficulty  of  his  duty 

technician  whose  aim  was  to  get  some  particular  is  increased  not  only  by  the  complexity  of  the  e- 

specialization  in  his  field.  quipment  he  has  to  cope  with,  but  very  often  by  the 

In  this  case  it  is  possible  to  get  positive  re-  particular  conditions  he  is  working  in,  which  ask 

suits  due  to  the  capacity  of  the  computer  to  sort  or  better,  force  him  to  solve  the  problems  in  a 

informations  and  to  show  them  to  this  special  stu-  time  as  short  as  possible. 

dent,  to  its  speed  in  evaluating  his  answers  and  To  reach  this  target  he  needs  what  we  can  re- 

to  change  accordingly  its  teaching  strategy,  toge-  sume  in  a  single  word;  experience, 

ther  with  the  limited  extent  of  the  subject.  But  Armed  Forces,  who  need  this  kind  of  skilled 

But  let  us  consider  another  figure:  just  to  technicians,  are  seldom  in  the  conditions  to  pre¬ 
keep  close  to  my  company's  experience,  and  to  my  pare  them,  due,  for  example,  to  service  reasons 

own,  too,  let  us  take  a  student  who  knows  nothing  which  often  compell  to  move  experienced  people  to 

about  electronics,  and  who  is  supposed  to  become  a  different  positions,  but  mostly  because  of  the 

technician  able  to  maintain  the  sophisticated  cir-  shortness  of  time  which  is  at  disposal  for  training 

cuits  of  a  Fire  Control  System.  together  with  the  necessity  to  send  people  to  their 

Even  if  from  a  theoretical  point  of  view  we  can  final  jobs  as  soon  as  possible, 
imagine  a  computer  able  to  manage  the  enormous 
quantity  of  informations  necessary  to  achieve  this 
goal,  we  have  to  face,  at  least,  two  problems:  the 


361 


'•  I 

t  ( 
I  I 


I  1 

f  ’i 


J 

•I 


F 


A  NEW  COMPUTER  APPROACH 

The  Computer  as  an  Element  of  the  Training  System. 

The  je  problems  cannot  find  a  satisfactory  solu¬ 
tion  by  trr  fional  training  methods,  neither  a 
computer  ca.  10  it  by  itself. 

But  really  it  can  be  done  in  an  easy  way  using 
a  computer  as  a  component ,  no  longer  as  "the  compo¬ 
nent",  of  a  training  system  including  other  ele¬ 
ments  which  a  computer  can  coordinate  and  control 
in  the  pro, per  way. 

Before  starting  the  analysis  of  the  Computer 
Aided  Training  System  we  are  introducing,  it  is 
better  at  this  point  to  have  a  step  backwards,  be¬ 
cause  the  system  is  the  result  of  an  evolution 
lasted  for  years,  and  to  understand  it  completely 
it  is  use  full,  perhaps  necessary,  to  have  a  look 
at  the  intermediate  levels  of  this  evolution,  which 
are  strictly  interconnected  with  the  development  of 
the  company  which  implemented  it. 

The  Gajon  Experimental  Programned  Learning  System 
for  Military  Technicians. 

The  Gajon  Institute  of  Technology,  established 
in  1949  and  since  the  very  beginning  involved  in 
military  technicians  training,  realized  the  neces¬ 
sity  of  studying  a  new  training  methodology  to 
solve  the  problem  of  preparing  skilled  manpower  at 
the  satisfactory  level  in  a  shorter  time,  and,  if 
it  was  possible,  at  a  lower  cost. 

A  basic  point  is  the  following  one:  the  student 
has  to  partecipate  to  the  learning  process  in  an 
active  way. 

This  active  behaviour  can  be  resumed  into  two 
aspects: 

1)  the  student  has  to  reach  the  fixed  level  of 
knowledge  working  according  to  his  learning 
rate 

2)  the  student  himself  must  be  considered  an 
interesting  teaching  resource,  both  from  the 
self  teaching  point  of  view  and  as  a  useful 
help  for  the  real  teacher. 

On  the  other  hand,  the  continuous  increase  of 
scientific  knowledge  to  be  assimilated  pushes  the 
students  to  accelerate  their  learning  rate;  but 
unfortunately,  very  often  the  learning  rate  is 
slower  than  the  transmission  of  knowledge. 

Deep  changes  in  the  methodology  are  therefore 
necessary  to  improve  teaching  productivity. 

Another  problem  is  the  growing  of  scholastic 
population,  and  the  increase  of  knowledge  in  quan¬ 
tity  and  quality  is  not  balanced  by  a  proportional 
increase  in  number  and  preparation  of  teachers. 

To  solve  the  problem,  we  have  to  work  towards 
two  directions:  a  better  U3e  of  the  teacher  as  di¬ 
dactic  means  and  the  realization  of  an  individualiz¬ 
ed  autonomous  teaching. 

These  considerations  together  with  the  study 
carried  on  learning  process,  suggested  to  adopt  a 
"learner  centered"  method. 

The  Gajon  answer  is  a  learning  philosophy  dif¬ 
ferent  from  the  traditional  one:  through  a  direct 
experimentation  the  student  is  pushed  to  observe 
and  to  analize  phenomena  to  reach  the  conclusions 
by  himself. 

On  the  contrary,  the  normal  training  method  is 
composed  by  theoretical  lessons  and  subsequent  ex¬ 
periments  to  apply  and  test  the  studied  concepts. 

With  the  Gajon  system  the  student  is  brought  to 
discover  by  himself  the  laws  regulating  the  pheno¬ 
mena  following  the  same  way  of  the  scientists;  it 
is  evident  that  in  such  a  way  the  student  will  un¬ 
derstand  completely  the  laws  and  will  be  able  to 


apply  them  without  forgetting  them. 

The  teacher'e  task  is  exalted  as  he  haa  not 
only  to  dictate  formulae  but  he  must  amplify  and 
deepen  the  concepts  and,  above  all,  help  the  stu¬ 
dent  to  develop  his  own  capacity  of  observation, 
synthesis  and  reaaonment,  the  real  bases  of  know¬ 
ledge. 

The  Gajon  Desk:  an  Open  System.  To  reach  these 
targets  a  new  training  medium  had to  be  designed 
as  most  of  the  training  systems  employ  pre-aasem- 
bled  block  circuits  which  become  obsolete  because 
of  the  fast  evolution  of  electronics. 

On  the  contrary,  the  Gajon  desk  is  an  open-end 
system  as  it  allows  the  greatest  liberty  in  assem¬ 
bling  every .kind  of  electronic  circuit,  also  the 
most  complex  ones,  employing  standard  components; 
on  the  desk  even  a  complete  TV  set  or  a  radar 
bearing  generator  can  be  assembled. 

All  these  things  are  tl.a  best  guarantee  that 
the  Gajon  system  will  never  become  obsolete,  sav¬ 
ing  customers'  investments. 

Another  confirmation  of  this  fact  is  that,  ex¬ 
cept  for  obvious  technical  improvements,  Gajon 
desk  haa  been  maintaining  for  30  years  the  same 
basic  principles  without  losing  validity,  and  the 
desks  built  in  the  1950s  are  still  employed  with 
good  results. 

Main  Characteristics  of  the  Gajon  Desk,  The 
desk  provides  a  work  surface  which  allows  to  as¬ 
semble  even  very  complex  circuits  without  solder¬ 
ing,  using  electronic  tubes,  transistors,  inte¬ 
grated  circuits,  microprocessors  and  any  kind  of 
other  components. 


1-  The  Gajon  desk  mod.  1001B,  designed  to  match 
the  necessities  of  a  modern  programed  learning 
experimental  method. 


The  work  surface  is  composed  by  amovable  plates 
which  may  be  taken  out  with  the  assembled  circuits, 
leaving  the  surface  free  for  other  students. 

The  desk  power  supplies  give  all  the  voltage 
values  necessary  to  make  the  circuits  work  through 
an  automatic  feeding  line  integrated  in  the  work 
surface. 

All  the  power  supplies  are  electronically  pro¬ 
tected  against  short  circuits;  an  acustic  alarm 
indicates  overcurrent  conditions  and  a  control  pa¬ 
nel  shows  the  overloaded  power  supply. 

A  special  turret  is  provided  to  keep  measuring 
instruments  and  signal  generators  which  are  modu¬ 
lar  and  plug-in  type. 

After  the  use,  instruments  and  components  can 
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be  stored  into  a  special  trolley  supplied  with  the 
desk . 


2-  Sample  of  the  special  emovable  assembling 
plates  which  form  the  work  surface  of  the 
Gajon  desk.  It  is  possible  to  see  here  the 
components  (mounted  on  standard  supports) 
and  the  special  connecting  elements. 


How  the  Compute t  Can  Improve  the  Ga.jon  Method. 

The  results  we  hr  'e  been  getting  for  twenty 
years  by  the  employ  of  our  very  special  programm¬ 
ed  learning  method  have  been  satisfactory,  but  as 
a  matter  of  fact,  in  the  educational  field  you 
cannot  pretend  to  have  reached  the  top,  and  that 
is  even  more  true  for  an  Institute  engaged  in  the 
training  of  electronic  military  technicians. 

On  the  other  hand,  we  realized  that  there  were 
some  problems  due  to  the  particular  market  we  were 
working  in:  very  often,  the  instructors  we  prepar¬ 
ed  in  our  Center  in  Genoa,  after  two  or  three 
years  had  to  leave  their  schools  for  other  desti¬ 
nations  because  of  changes  in  their  careers  and  it 
was  not  easy  to  replace  them. 

In  our  training  philosophy,  the  equipment  are 
important,  but  even  more  important  is  the  method 
to  use  them,  and  this  is  something  that  only 


instructors  prepared  by  us  know;  in  our  after¬ 
sale  services  training  for  several  groups  of  in¬ 
structors  is  included,  but  it  seldom  happens  for 
a  lot  of  reasons  connected  to  the  rigidity  of  mi¬ 
litary  organizations. 

Therefore,  we  started  trying  to  add  something 
to  the  equipment  to  make  easier  for  the  new  in¬ 
structor  to  follow  our  training  method. 

The  solution  of  the  problem  could  not  have  been 
a  simple  manual,  but  it  had  to  be  something  alive 
and  able  to  guide  the  instructor  step  by  step 
keeping  him  close  to  the  right  procedure. 

But  we  were  also  looking  for  something  so  flex¬ 
ible  to  let  the  experienced  instructor  modify  the 
medium  itself  according  to  its  philosophy,  once 
he  got  the  complete  control  on  the  system. 

We  found  out  that  a  properly  designed  computer 
could  give  a  positive  solution  to  our  investiga¬ 
tions  . 

Of  course,  the  possibility  offered  by  the  com¬ 
puter  added  to  the  basic  desk  to  avoid  problems 
connected  to  the  change  of  instructors  has  not 
been  the  sole  reason  why  we  took  that  decision. 

As  a  matter  of  fact,  the  computer  aided  train¬ 
ing  is  a  logical  improvement  of  an  experimental 
programmed  learning  system  as  the  one  we  develop¬ 
ed;  it  adds  to  the  system  several  advantages: 

-  it  makes  it  easier  to  have  a  wider  possibili¬ 
ty  of  ramifications,  with  a  consequent  in¬ 
crease  of  self-adapting  capability  and  perso¬ 
nalization  of  the  course  itself 

-  the  informations,  can  be  automatically  stored 
and  managed 

-  through  special  algorithms  the  computer  can 
satisfy  requests  which  ask  for  elaboration 
(like  execution  of  mathematical  operations) 

-  with  some  restrictions,  answers  freely  ex¬ 
pressed  can  be  accepted  and  evaluated 

-  it  is  possible  to  generate  files  of  data, 
having  a  fast  access  to  use  or  to  correct 
them.  This  is  an  important  condition  to  have 
the  necessary  feedback  in  the  program:  the 
changes  which  experience  may  suggest  are  not 
so  easy  to  be  done  if  the  training  means  is 

a  manual  or  a  teaching  machine 

-  taking  into  account  the  previous  curriculum 
(preliminar  test,  results,  previous  answers, 


3-  The  Master  Desk  Mod.  2000M  tegether  with  its  Accessories. 
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etc.)  automat  ically  stored  in  the  memory,  it 
is  possible  to  adapt  the  sequence  ol  the 
course  to  students'  characteristics. 

Due  to  the  modularity  of  our  programs  and  to 
the  step  by  step  procedure  of  our  courses  it  did 
not  seem  difficult  to  modify  t lie ni  slightly  to 
transler  them  in  a  computer  memory. 

Moreover,  we  wanted  to  take  the  advantages  of¬ 
fered  by  the  development  of  audiovisual  aids,  as 
well;  our  computer  should  have  had  the  possibility 
to  manage  a  complete  audiovisual  set,  integrated 
in  the  classroom,  including  CC’TV  system,  video¬ 
slides,  VCR  and  automatical Ly  operated  by  the  com¬ 
puter  at  the  proper  moment  of  the  course. 

Taking  into  account  all  these  requisites  we 
had  to  make  a  first  important  choice:  a:-,  intelli¬ 
gent  terminal  for  each  student  connected  to  a  main 
computer  managing  the  complete  classroom  or  the 
whole  school,  or  an  individual  minicomputer? 

We  prefered  the  second  possibility,  as  the  best 
one  from  the  economical  point  of  view  and  due  to 
its  flexibility. 

We  could  not  find  on  the  market  a  minicomputer 
matching  all  our  requirements,  so  wo  designed  our 
own  one;  we  solved  a  Lot  of  technical  problems, 
but  we  think  that  the  results  can  be  defined  sa- 
ti sfactory. 

Cajon  Computerized  Training  System. 

In  a  classroom  equipped  by  our  computerized 
training  systen.  we  have  two  different  types  of 
desk:  a  master  desk  for  the  instructor  and  the 
student  desk,  each  one  computerized. 

In  the  steel  structure  of  the  student  desk  is 
included  a  complete  computer  assisted  electronic- 
laboratory. 

The  student  has  at  his  disposal:  the  computer 
with  its  keyboard,  an  automatic  feeding  system, 
a  complete  set  of  plug-in  instruments,  a  special 
work  surface  where,  as  in  the  previous  model,  he 
can  assemble  without  any  soldering  any  kind  of 
electronic  circuit,  starting  from  the  simplest  up 
to  the  most  advanced  ones,  using  the  standard 
components  which  are  stored  in  the  utility  trolley. 


The  Master  Desk  Mod,  2 POOH.  The  desk  2000M 
for  the  teacher  has  exactly  the  same  steel  struc¬ 
ture  as  the  student  desk,  but  it  is  missing  all 
circuit  assembling  systems,  circuit  feeding  sys¬ 
tem,  instrumentation  and  components. 

On  the  front  panel,  instead  of  instruments,  al 
digital  interfacing  with  the  student  desks  and  a 
9"  monitor  used  by  the  teacher  to  preview  audio¬ 
visual  programs  are  included. 

On  the  surface  there  are  two  keyboards,  the 
computer,  cassette  recorder/player  and  a  transpa¬ 
rent  crystal  used  ns  an  overhead  projector  thanks 
to  the  CCTV  camera  placed  on  top  of  the  crystal. 

Slides  are  projected  on  the  same  crystal  from 
the  slide  projector  placed  underneath,  allowing 
the  pick-up  from  the  CCTV  camera. 

In  the  trolley  beside  the  desk  the  video  cas¬ 
sette  recorder  is  mounted,  and  a  storage  area  is 
provided  for  video-cassettes,  transparencies  and 
slides  cartridges. 

The  desk  includes: 

-  front  panel  with  all  controls 

-  12"  CRT  video  graphic  alphanumeric  display 

-  9"  CRT  monitor 

-  alphanumeric  keyboard 

-  electronic  symbols  graphic  keyboard 

-  computer  and  interfacing  facilities 

-  computer  cassette  recorder/player 

-  intercom 

-  CCTV  camera 

-  slide  projector 

-  video  cassette  recorder 

-  computer  printer 

Monitor  and  computer  are  plug-in  modules,  al¬ 
lowing  quick  substitutions  in  case  of  breakdown. 

The  computer  is  provided  by  an  autodiagnosis 
board  to  reduce  trouble-shooting  time. 

There  are  some  differencies  between  the  stu¬ 
dent's  computer  (that  we  call  CAT/S)  and  the  tea¬ 
cher's  one  (CAT/M). 

The  Computer  for  the  Student's  Desk  (CAT/S). 

The  CAT/S  is  a  dual  processor  microcomputer  sy 
stem  with  a  dual  on-board/off-board  multiple  bus 


structure  that  allows  each  processor  to  use  its 
own  memory  and  input/output  without  utilizing  a 
conmon  system  bus. 

A  common  system  bus  is  used  only  when  either 
processor  requires  access  to  the  common  memory. 

The  system  generates  bus  busy  and  bus  reque¬ 
st  signal.  Under  these  conditions  two  indepen¬ 
dent  programs  or  data  exchange  programs  can  be 
run  simultaneously. 

Both  processors  can  accept  parallel  data  from 
any  eight  bit  bus  peripherals. 

Serial  I/O  via  UART  is  provided.  Easy  user 
access  is  provided  via  multipins  connectors  to 
data  and  address  buses.  All  control  signals  are 
accessible. 

Main  features: 

-  8080A  processor 

-  crystal  controlled  clocks 

-  crystal  controlled  baud  rate  for  cassette 
load 

-  crystal  controlled  composite  sine  pulse  for 
CRT  display 

-  TTL  compatible  input/output 

-  30  Kbyte  of  static  RAM  memory 

-  system  control  program  stored  in  ROM 

All  components  are  mounted  on  fiberglass 

through-hole-plated  plug-in  boards  of  double  Euro¬ 
size  with  gold  plated  edge  connector;  each  board 
is  mounted  on  a  mother  board  with  gold  plated  con¬ 
nector  sockets. 

All  boards  are  housed  in  a  metal  cabinet  with 
forced  air  cooling  system.  Connections  between 
computer  and  peripherals  are  made  via  flat  ribbon 
cable. 

The  following  peripherals  are  standard  in  the 
basic  student  desk: 

-  cassette:  standard  audio  playback  unit  with 
remote  computer  control.  Tape  speed  is  4.75 
cm/sec..  No  special  cassette  tapes  are  re¬ 
quired. 

-  printer:  matrix  inpact  with  ink  ribbon.  No 
special  paper  is  required,  Paper  width  8". 

-  monitor:  standard  12"  black  and  white  video 
display  with  antireflection  screen.  Bandwidth 
not  less  than  8MHz.  Composite  video  signal. 

-  keyboard:  alphanumeric  keyboard  ASCII  encoded. 
Cursor  control  and  special  function  key. 

The  Computer  for  the  Instructor's  Desk  (CAT/M). 

The  basic  microcomputer  system  of  CAT/M  is  the 
same  as  CAT/S. 

Additional  software  control  functions  allow  the 
teacher  to  generate  special  graphic  characters,  to 
prepare  lessons  and  to  dump  them  on  cassette  tapes. 

The  CAT/M  contains  audio  cassette  playback  and 
record  facilities. 

The  teacher  has  at  his  disposal  VCR  and  slide 
projector  combined  with  telecamera  as  additional 
facilities  for  interconnections  between  his  desk 
and  the  display  units  of  any  or  all  students. 

Audiolinks  enable  contacts  between  teacher  and 
any  or  all  students  simultaneously. 

The  following  peripherals  are  standard  in  the 
master  desk: 

-  cassette:  standard  audio  playback/record  unit 
with  remote  computer  control.  Tape  speed  4.75 
cm/sec..  No  special  cassette  tapes  required. 

-  monitors:  same  as  monitor  of  CAT/S  used  as  vi¬ 
deo  monitor  or  student's  control  monitor.  Pre¬ 
view  monitor  for  VCR,  telecamera  and  slide 
projector. 

-  VCR:  black  and  white  and  colour  with  three  in¬ 
ternational  standards  of  decoding.  All  func¬ 


tions  are  remote  controlled  by  the  teacher 
or  by  the  students'  computer  according  to  the 
lesson  program. 

-  slide  projector:  standard  Kodak  Carousel. 

-  telecamera:  Vidicon  interlaced  1/2"  camera 
of  medium  brightness  type;  bandwidth  10MHz 
giving  a  resolution  of  700  lines. 

-  keyboards:  ASCII  encoded  alphanumeric  key¬ 
boards;  special  graphic  keyboard;  cursor  con¬ 
trol  and  special  function  keys. 

-  audio  communications  system:  low  power  duplex 
speaker  impedance  600  ohms. 

A  very  interesting  feature  offered  by  our  mas¬ 
ter  computer  is  the  possibility  for  the  teacher 
to  modify  the  standard  programs  we  supply  together 
with  the  system,  or  to  write  brand  new  programs 
even  if  he  is  not  a  skilled  computer  programmer. 

To  make  it  possible  we  developed  a  very  simple 
programming  language  and  to  solve  the  problems 
usually  connected  to  graphic  programs,  we  designed 
a  special  graphic  keyboard. 

To  each  key  of  the  last  a  graphic  electronic 
symbol  or  a  part  of  a  symbol  are  corresponding  in 
the  computer  memory. 

If  the  teacher  reeds  an  electronic  symbol  to 
draw  a  circuit  on  the  screen,  what  he  has  to  do  is 
simply  to  press  the  key  corresponding  to  it. 

If  the  symbol  he  needs  is  composite,  it  means 
composed  by  several  parts  on  different  keys,  he 
can  press  a  special  sequence  key  corresponding  to 
that  symbol  and  automatically  the  computer  shall 
collect  the  single  parts  to  draw  the  complete 
symbol . 

Two  symbols  correspond  to  each  key;  the  choice 
is  made  by  pressing  one  of  the  two  function  keys, 
the  yellow  or  the  green  one. 

The  Gajon  Computerized  Training  System  at  Work. 

The  philosophy  of  the  training  method  is  un¬ 
changed:  the  student  has  to  work  on  the  desk,  mak¬ 
ing  his  own  practical  experiences  and  reaching  the 
laws  which  regulate  the  phenomena  correlating  his 
observations  and  the  measurements  he  can  take. 

At  any  step  the  student  is  guided  by  his  own 
computer. 

In  fact,  as  we  told  above,  each  student's  desk 
is  provided  by  a  microprocessor  with  alphanumeric 
keyboard,  display  and  cassette  player;  the  lesson 
is  stored  on  the  cassette,  ready  to  be  transferred 
to  the  computer  memory. 

The  instructions  for  the  student  and  the  gra¬ 
phical  representation  of  the  circuit  he  has  to  as¬ 
semble  appear  on  the  screen  controlled  by  the  com¬ 
puter. 

This  special  display  developed  by  Gajon  can  be 
used  as  a  graphic  terminal  for  the  computer  and 
also  as  a  normal  video  screen  or  the  audiovisual 
aids. 

Through  the  keyboard  the  student  gives  the  data 
to  the  computer  about  the  measurements  he  has  done 
on  the  circuit,  answering  the  questions  the  compu¬ 
ter  asks  him. 

In  the  student  computer  memory  the  complete 
sequence  of  the  experiments  has  been  stored  toge¬ 
ther  wit’’  the  numerical  results  of  the  experiments; 
the  stuc  must  perform  them  according  to  the  in¬ 
structions  received  from  the  computer. 

A  special  program  makes  the  comparison  between 
students'  results  and  the  right  ones  in  a  range  of 
approximation. 

If  the  student's  answer  is  correct,  through  the 
display  the  computer  shows  to  him  the  next  experi¬ 
ments  to  be  performed;  in  the  opposite  case  it 
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YELLOW  SPEC! HI.  CHRHRCTEH8  KIM  LAYOUT 


RODE 


1  2  3  4  5  6  7  8  9  10  11  12  13 

0000000000000 

14  IS  16  17  18  19  20  21  22  23  24  25  26 

00000000000000 

27  28  29  30  31  32  33  34  35  36  37  38  39 

@000000000000 

40  4  1  42  43  44  45  46  47  48  49  50  51  52 

00000000000000 

53  54  55  56  57  58  59  60  61  62  63 

l:M<IHB000000 


GRECH  SPECIAL  CHARACTERS  KEYBOARD  LAYOUT 

1  2  3  4  5  6  7  8  9  10  11  12  13  — . 


MODE 


r 


000000 B0000D0 

14  IS  16  17  18  19  20  21  22  23  24  25  2$ 

L  BDODDDDDODDDDO J 

27  28  29  30  31  32  33  34  35  36  37  38  39 

□000000000000 

40  41  42  43  44  45  46  47  48  49  50  51  52 

@0000000000000 

53  54  55  56  57  58  59  60  61  62  63 

□□□□□□□□□□□ 


Special  Sequences:  GREEN  1  to  26 


MODE 


1=  Transistor 

2=  Vert. line 

3=  Vert . Resistor 

1 

4=  Coil  ^ 

5-  Coil  ^ 

6-  Iron  Core  1 

7-  Hor. Resistor  /A*- 

8-  SCR 

S--  Box  □ 

5-6-7-  Computer  Printed  Graphic.  Keyboard  Map. 
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Circuit  Drawings  -  Examples  MODE 


8-  Sample  of  circuit  drawings  executed  by  the  CAT/M  under  special  graphic  keyboard  control. 


18.2.1.1  SCR  en  Cfl,  Control  mn  CC  (Revision  i)  MODE 


Real izor  el  circuito 
Cuondo  VgaVmin  w  Va=50v .ob¬ 
server  la  forma  de  onda  on 
1  os  puntos  ft  y  B. 

Rumentar  lentamente  Vg  Kosta 
quo  el  SCR  mmp  i  era  a  func te¬ 
nor. Repet  ir  la  observacion 
del  punto  2),medir  con  ml  te- 
ster  la  caida  de  tension  err- 
tre  los  term inales  de  R. 
Graduar  Vg  al  maxi mo  valor 
de  tension,  repet ir  la  obser- 
yacion  u  medir  la  caida  de 
tension  en  R. 


Rl=  6.8KB  Pl  =  5009  Tester  COHCLU8IOHES . . 

R2=  2  2Kfl  Cr 1= 1N4007  Oscil.  El  SCR  en  CR,  controlado  apli- 
R3=  4709  SCR =$4003  cando  al  GRTE  una  tension  contt- 
R4=  47K9  Iristrurnento  1111  nua  permite  regular  la  corriente 
RS=  1009  ''  HI?  en  la  carga  variando  la  Vg. 


9-  Sample  of  experiment  sheet  printed  by  CAT/S. 
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asks  the  student  to  check  again  the  circuit,  to  set 
it  up  in  the  correct  way. 

If  the  results  remain  incorrect,  automatically 
the  computer  sends  the  student  hack  to  the  point  of 
the  previous  lesson  he  did  not  understand. 

At  the  same  time  the  machine  keeps  the  score  of 
the  student,  which  will  be  a  part  of  his  curriculum 

To  complete  the  subject  the  student  is  study¬ 
ing,  the  computer  automatically  shows  to  him  short 
films  or  slides  on  the  display. 

The  complete  system  is  controlled  by  the  master 
desk. 

At  any  moment  the  instructor  can  copy  on  his 
screen  what  is  appearing  on  student's  one  and  vice- 
versa;  he  can  also  verify  the  individual  score  or 
the  score  of  the  complete  class  to  check  the  gene¬ 
ral  behaviour. 

At  the  end  of  the  lesson  the  conclusions  writ¬ 
ten  by  the  student  using  his  keyboard  are  automati¬ 
cally  transferred  to  the  teacher,  to  let  him  check 
them. 

In  such  a  way  the  teacher  gives  the  final 
judgement  about  student's  behaviour  if  it  cannot  be 
resumed  by  a  simple  numeric  answer. 

As  the  lesson  goes  on,  everything  the  student 
is  writing  is  printed  on  paper  automatically,  so 
that  at  the  end  of  the  course  each  student  will 
have  his  personalized  manual. 

All  the  system  is  continuously  checked  by  an 
autodiagnosis  apparatus,  allowing  a  fast  fault¬ 
finding  and  replacing  of  the  damaged  boards. 

Behind  the  previously  remembered  ones,  the  ad¬ 
vantages  of  the  system  are  very  interesting. 

We  have  a  program  including  all  audiovisual 
aids,  allowing  the  student  to  proceed  at  his  own 
pace,  according  to  his  learning  capability;  the 
proceeding  of  the  course  is  the  best  available,  be¬ 
cause  it  has  been  prepared  by  the  Gajon  specialized 
instructors. 

In  this  way  each  student  has  an  instructor  at 
his  own  disposal,  the  computer.  Teacher's  function 
is  not  cancelled,  but  exalted,  because  his  aim  will 
be  to  help  the  student  where  the  computer,  as  a 
machine,  cannot  succeed  without  being  bothered  by 
routine  explanations;  it  will  not  be  necessary  to 
keep  the  marks  of  each  student,  because  this  is  done 
by  the  computer. 

The  time  the  teacher  can  use  to  perform  new 
researches  has  been  increased  and  he  can  modify  the 
program  whenever  it  is  needed. 

In  fact,  as  we  told  above,  he  can  change  the 
programs  in  the  computer  memory  simply  typing  on 
the  keyboard  the  new  lesson,  even  if  he  does  not 
know  anything  about  computer  programing,  due  to 
thespecial  microprogrammed  keyboard  he  has  got  at 
his  disposal. 

Courses  which  Can  Be  Performed  by  the  Gajon 
System.  The  Gajon  desk  with  the  components,  the 
instruments  and  the  computer  programs  supplied  in 
the  basic  version  allows  to  perform  the  following 
courses: 

-  Basic  Electricity 

-  Basic  Electronics 

-  Basic  Digital  Electronics 

-  Basic  Communications 

Moreover,  accoridng  to  customers'  necessities, 
there  are  some  standard  advanced  courses;  to  per¬ 
form  them  additional  components  and  instruments  are 
needed  to  complete  the  basic  desk  equipment. 

They  are  covering  the  following  subjects: 

-  Electrical  Components  and  Circuits 

-  Electrical  Machines 


-  Logic  Circuits 

-  Basic  and  Advanced  Computers 

-  Electrical  Machines  and  Controls 

-  Synchro  and  Servosystems 

-  B/W  Television 

-  Colour  Television 

-  Antennae 

-  Microwaves 

-  Radar 

-  Sonar 

-  Pulse  Communications 

-  Wire  Communications 

-  Gyrocompass 

Of  course,  thanks  to  the  flexibility  of  the 
system  and  to  the  experience  of  our  company,  new 
programs  can  be  designed  by  our  specialized  in¬ 
structors  according  to  customers'  requirements. 

Advantages  of  the  Gajon  Computerized  Training 
Systems ■ 

Beyond  the  advantages  offered  by  other  computer 
aided  programed  learning  methods,  the  Gajon  sy¬ 
stem  offers  the  following  ones: 

Coherence  -  It  is  a  big  advantage  to  utilize 
the  same  didactical  system  from  basic  electricity 
up  to  the  most  advanced  specializations,  because 
in  such  a  way  every  course  gives  exactly  the  know¬ 
ledge  required  for  the  following  ones,  without 
useless  repetitions,  jumps  and  dangerous  changes 
in  the  methodology. 

Cost 8  -  To  have  a  unique  system  covering  dif¬ 
ferent  specializations  avoids  the  necessity  to 
have  multiple  equipment  and  laboratories,  with  an 
evident  saving.  The  additional  kits  are  adding 
only  the  components  and  the  specific  instruments 
for  each  technique  utilizing  the  same  space,  the 
feeding  system,  the  standard  instruments  and  the 
other  facilities  offered  by  the  basic  desk. 

Up-to-dating  possibilities  -  As  it  is  of  open- 
end  type,  the  Gajon  system  is  always  efficient 
because  even  the  new  techniques  which  will  be  de¬ 
veloped  in  the  future  can  be  integrated  in  it. 

That  happens  also  thanks  to  the  modularity  of 
the  program  which  lets  new  lessons  be  inserted. 

Flexibility  -  Even  if  each  experiment  is  follow 
ing  a  careful  planif ication  according  to  a  pro¬ 
grammed  learning  system,  the  instructor  who  con¬ 
siders  it  convenient  can  design  different  experi¬ 
ments,  as  the  system  allows  it. 

Training  time  -  One  of  the  most  evident  advan¬ 
tages  of  the  Gajon  system  tor  Military  Schools  is 
the  reduction  of  the  training  time,  without  com¬ 
promizing  the  quality  of  the  result. 

Realization  of  special  courses  -  The  Gajon  In¬ 
stitute  can  modify  the  standard  programs  according 
to  customers'  requirements.  This  service  is  free 
of  charge  and  lasts  for  ten  years  to  guarantee  the 
continuous  matching  of  the  system  with  school 
needs . 

Instructors'  training  -  The  cost  of  the  equip- 
ment  is  including  a  one  month  training  period  for 
the  instructors  in  Genoa  Gajon  Center,  with  board 
and  lodging  to  give  them  the  practical  experience 
which  is  necessary  to  get  the  best  results  from 
the  system. 

Furthermore: 

-  it  allows  a  self-tailored  training;  each  stu¬ 
dent  proceeds  according  to  his  capacity 

-  every  student  can  call  the  teacher  at  any  mo¬ 
ment  without  disturbing  his  fellows 

-  it  is  not  as  cold  as  the  other  computer  aid¬ 
ed  training  systems  because  it  is  interactive  and 
teacher's  presence  is  not  cancelled 
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-  it  allows  continuity  in  the  methodology  even 
when  the  teacher  is  replaced 

-  the  instructor  can  dedicate  more  time  to  each 
student  because  the  routinework  is  done  by  the 
computer 

-  it  allows  the  continuous  check  of  the  class 
eliminating  the  needs  of  periodical  examinations 

-  at  the  end  of  the  course  the  student  will  get 
a  complete  personalized  manual,  automatically 
printed  by  the  computer  during  each  lesson 

-  it  allows  an  automatic  and  continuous  commu¬ 
nication  between  students  and  instructor 

-  the  flexibility  of  the  system  allows  also  to 
graduate  the  dec  ness  in  which  every  subject  has 
to  be  studied.  _n  fact,  the  same  circuit,  for 
eample,  an  operational  amplifier,  can  be  studied 
at  informative,  technical  or  university  level.  The 
choice  can  be  done  at  the  beginning  of  the  course, 
when  a  lot  of  elements  must  be  defined:  targets 
and  plan  of  the  course  itself,  type  and  required 
output  level  of  the  personnel,  number  of  necessary 
people  to  cover  service  needs,  time  at  disposal. 

Final  Considerations. 

The  computerized  system  we  presented  above  is  a 
brandnew  product,  and  therefore  we  cannot  pretend 
to  have  in  one-year  experience  on  the  field  con¬ 
clusive  data  about  its  success. 

Anyway,  we  have  to  take  into  account  that  it 
applies  and  improves  the  method  we  have  been  uti¬ 
lizing  for  more  than  30  years  and  which  have  been 
carefully  tested  in  several  occasions. 

The  most  important  test  consisted  in  comparing 
in  the  various  countries  two  different  groups  of 
students. 

The  first  group  included  students  who  had  been 
taught  according  to  traditional  methods  while  the 
second  group  included  students  trained  with  Gajon 
methods. 

The  first  noticeable  fact  was  that,  even  though 
the  initial  level  of  instruction  of  the  students 
taught  with  Gajon  method  was  inferior  to  that  of 
the  other  group,  it  was  necessary  to  teach  the 
students  following  traditional  methods  for  a  pe¬ 
riod  of  22  weeks  as  against  only  9  weeks  with  the 
Gajon  method. 

Traditional  method  trained  students'  final  per¬ 
centage  of  passes  was  28Z. 

Gajon  method  trained  students  achieved  70Z. 

A  much  higher  percentage  of  passes  has  been  re¬ 
gistered  when  students  began  their  initial  prepa¬ 
ration  with  the  Gajon  system. 

For  what  Computer  Aided  Training  System  is  con¬ 
cerned,  the  first  courses  are  now  in  progress  in 
different  countries  and  we  can  affirm  that  the  par¬ 
tial  checks  we  are  making  are  showing  what  we  ex¬ 
pected,  confirming  the  advantages  of  the  CAT  com¬ 
bined  with  the  Gajon  programmed  learning  training 
system. 
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"TRAINING  WITH  A  SHORTENED  RANGE  CARTRIDGE 
FOR  AUTOMATIC  RIFLE" 


JEAN  CHEVALIER  (Colonel  S.T.A.T.) 

FRANCOIS  AMBROSI  (Ing6nleur  en  Chef  de  l'Aroement  D.T.A.T.) 


ABSTRACT 


In  order  to  allow  for  firing  manoeuvre,  on  the  ground,  of  the  combat  group  In  reasonably 
safe  conditions,  a  shortened  range  firing  system,  adaptable  to  the  French  automatic  rifle,  has 
been  developed. 

In  a  first  part,  after  having  recalled  the  military  requirements,  the  system’s  conditions 
of  use  are  exposed,  and  in  particular  those  of  its  associated  5,56  ammunition. 

The  second  part  is  devoted  to  the  description  of  the  materiel  which  has  been  studied  on 
the  ground  of  the  military  specifications  Issued  from  the  requirements  stated  before. 

The  materiel  is  composed  of  a  kit  adaptable  to  the  rifle  and  of  a  5.56  cartridge  with  a 
plastic  bullet  said  "balplast" . 

This  system  Is  at  the  last  stage  of  Its  development  and  should  be  proposed  for  evaluation 
to  the  official  services  by  the  end  of  1981. 


INTRODUCTION 


When  adopting  at  the  beginning  of  the  century 
high  velocity  small  calibre  infantry  weapons,  the 
French  Array  necessarily  grew  Interested  in 
shortened  range  training  systems  (Cf.  fig.  1). 

Whereas  service  ammunition  requires  heavy 
infrastucture  (firing  ranges)  and  large  size 
layouts  (safety  limits)  which  are  more  particularly 
unpermlsslble  In  Europe,  use  of  shortened  range 
ammunition  brings  forth  notable  reduction  of 
utilisation  restraint  and  cost. 

In  particular.  In  as  iduch  as  it  has  very 
short  lethal  range,  such  ammunition  affords 
collective  practice  on  the  training  ground,  and 
firing  manoeuvre  for  combat  groups,  which  cannot  be 
thought  of  with  service  ammunition. 

Widespread  adoption  throughout  the  past  ten 
years  of  automatic  individual  weapons  ipso  facto 
led  to  the  necessity  of  automatic  fire  practice. 

In  order  to  meet  this  particular  military 
requirement,  viz.,  "combat  group  training  to 
automatic  fire  in  open  ground",  a  special  shortened 
range  system  has  been  elaborated  which  can  be 
adapted  without  any  tooling,  by  sub-asserably 
replacement,  on  the  service  rifle. 


For  obvious  medium  range  accuracy  and  safety 
reasons,  the  22  LR  system  was  left  aside  and  an 
original  solution  was  preferred,  consisting  of  a 
lightened  body  and  a  specially  designed  round,  with 
a  training  purpose  bullet. 

The  round  is  derived  from  the  service 
ammunition  5.56  cartridge  case,  with  a  rebated  rim 
diameter . 

A  very  light  plastic  bullet  (10  grains 
boattail)  is  propelled  at  4,000  ft/sec  muzzle 
velocity  by  12  grains  of  fast  powder,  with  external 
ballistics  matching  exactly  that  of  the  service 
ammunition  bullet  up  to  100  meters. 

The  characteristics  of  the  ammunition  and  its 
subcomponents  are  described  In  detail. 

The  French  automatic  rifle  can  be  classed  as 
a  delayed  blowback  type.  With  respect  to  the  low 
impulse  available,  the  weight  of  moving  parts  haJ 
to  be  cut  down  consistenly  so  as  to  afford 
automatic  firing  mode. 

The  lightened  training  subcomponents  are 
supplied  as  a  kit  comprising  the  breech  block,  the 
bolt  he?.d  carrier  and  the  delay  lever,  directly 
interchangeable  with  the  respective  parts  of  the 
service  weapon. 


Fig.  I  The  shortened  range  cartridges  family  since  1900 
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In  ord«c  to  forbid  use  of  service  ammunition 
(most  (inadvisable  both  for  the  gunner  and  the 
people  around)  during  a  practice  session,  the 
diameter  of  the  recess  In  the  bolt  head  of  the  kit 
was  reduced  so  that  only  a  special  profile  balplast 
cartridge  with  a  rebated  rim  can  be  fired. 

MILITARY  REQUIREMENTS 

In  fact  the  military  need  Is  not  new,  but  It 
has  been  modlfided  with  the  widespread  use  of 
automatic  assault  rifles. 

In  France,  the  need  for  a  shortened  range 
system  able  to  work  In  the  automatic  mode  has  been 
confirmed  by  the  adoption,  In  July  1977,  of  the 
FAMAS  5.56  FI,  and  the  wish  to  use  fully  for 
training  the  potentialities  of  the  new  5.56  weapon 
system. 

Military  requirements 

The  external  ballistic  characteristics  of  the 
service  ammunition  are  such  as  to  lead,  on  the  one 
hand,  to  Important  safety  space  requirements,  on 
the  other  hand,  In  the  case  of  a  use  inside  a 
military  enclosure,  to  non  negligible  structures. 

According  to  these  safety  space  requirements, 
the  firing  grounds  have  to  be  very  large,  and  so 
are  generally  quite  distant  from  the  barracking  of 
the  troops,  this  heads  to  frequently  Important 
moving s. 

Besides,  In  the  camps  used  in  France  for 
manoeuvres  as  well  as  for  firing,  the  execution  of 
the  firing  course  necessary  for  the  Infantry  troops 
individual  and  collective  training  makes  It 
compulsory  to  neutralize,  for  safety  reasons,  wide 
spaces  which  are  therefore  not  available  for  the 
tactical  training  of  the  troops 

It  was  therefore  necessary  to  elaborate  an 
ammunition  having  the  same  ballistic 
characteristics  as  the  service  ammunition,  but  with 
a  shortened  range  (1,000  m) 

This  ammunition  makes  possible  :  the  use  of 
light  structures  Inside  the  barracks. 

The  performance  of  Instinctive  fire  practice 
without  blocking  more  space  than  la  necessary. 

Use  of  5.5o  balplast  cartridges 

Due  to  Its  characteristics,  the  balplast  cartridge 
has  the  same  external  ballistics  as  the  service 
ammunition  between  0  and  100  m,  but  its  terminal 
efficiency  Is  very  much  reduced  beyond  this 
distance  therefore  a  simple  brick  wall  stops  it 
beyond  100  m. 

The  safety  area  Is  limited  to  500  m  (instead  of 
3,500  for  the  service  ammunition). 

This  ammunition  Is  used  for  the  soldier's 
elementary  training,  the  time  of  which  being  very 
short  during  the  12  months  "Service  National".  The 
movings,  which  involve  delays  and  wastes  of  time, 
are  limited  to  a  minimum,  as  the  firings  take  place 
inside  the  barracks  of  the  troops. 


For  firing  with  a  cine  simulator  In 
enclosures  destined  to  this  purpose 
It  Is  therefore  necessary  that  the  functions  held 
by  the  service  ammunition,  In  particular  the 
automatic  firing,  be  possible  with  this  shortened 
range,  yet  real  firing  ammunition. 

THE  BALPLAST  KIT 

Therefore,  in  order  to  meet  the  requirement 
corresponding  to  the  tactical  use  of  the  automatic 
rifle  FAMAS  5.56.  a  shortened  range  firing  system 
has  been  developed. 

a)  it  has  been  STUDIED  on  the  ground  of 
military  requirements  the  priorities  of  which  are 
recalled  there  after. 

1.  Safety  : 

-  Dangerous  range  limit  Inferior  to  600  m 

-  The  kit  design  must  not  allow  firing  with 

service  ammunitions. 

-  The  pieces  must  be  easily  recognized  as 

well  as  the  ammunitions. 

2.  Firing  : 

-  the  performances  are  judged  In  comparison 
with  those  observed  for  the  rifle  with 
service  ammunitions,  on  the  same  targets, 
at  the  same  ranges. 

-  In  various  respects,  this  also  concerns  : 

-  probability  of  hit 

-  the  accuracy  (group  and  zero)  In  the 
ease  of  firing  with  a  rest  (bipod,  or 
sand  bag,  and  so  on...) 

3.  Reliability  s 

-  adverse  conditions 

-  climatic  extremes 

-  fouling 

-  miscellaneous  incidents 

4.  Life  expectation  : 

15,000  rounds  per  kit  (with  the  possibility 
of  exchanging  the  striker,  springs,  ejector, 
extractor. . .) . 

b)  It  is  COMPOSED  of  : 

-  an  adaptable  kit  for  the  whole  bolt  wit 

-  a  cartridge  with  a  plastic  bullet. 

-  a  magazine  (which  Is  In  fact  polyvalent, 
as  it  allows  the  rifle  to  work  with  any 
of  the  adopted  cartridges  :  service 
ammunitions,  blank,  dummy). 

c)  it  ENSURES  the  different  firing  modes  of 

the  rifle,  with  5.56  plastic  bullet 

cartridges,  and  a  useful  range  of  100  m. 
Fundamentally,  the  problem  to  solve  Is  not  a 
simple  one  .Automatic  firing  requires  a  lot  of 
energy,  and  that  is  a  priori  at  the  opposite 
of  shortened  range, let  alone  with  the 

requirements  about  accuracy. 


For  the  infantry's  Individual  as  well  as  collective 
further  training,  with  as  small  safety  areas  as 
possible. 
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Nevertheless ,  the  question  once  put,  the 
firsldees  to  come  forth  In  answer  were  these  ! 

For  the  ammunition,  try  to  scale  down  and 
adapt  for  the  5.56  automatic  rifle,  the  existing 
cartridge  used  for  the  7.5  rifle,  and  In  particular 
Its  projectile. 

As  for  the  weapon,  considering  the  low  energy 
available  with  the  balplast,  design  a  blowback 
unit,  the  FAMAS  being  a  priori  suited  for  this 

The  first  Ides,  l.e.  reduction  of  the  7.5 
round,  misfired,  so  to  say...  because  of  the  shape 
of  the  flat  based  bullet.  As  for  the  second  idea, 
It  was  quickly  dismissed  for  two  major  reasons  :  a 
technical  one  and  a  human  one. 


and  Pa  “  momentum  of  the  single  equivalent 

moving  part,  (for  breech,  bolt  carrier...) 

we  find  :  Pa  “  ~  M  Pp  and  Pg»(/i  — /  )  Pp 

Another  relation  between  the  mass  of  the 
single  equivalent  moving  pert  (  mr  )  and  the  mass 
of  the  moving  parts  (  )  for  the  breech  and 

(  mbc  )  for  the  bolt  carrier  Is  obtained  from  the 
conservation  of  energy  :  (Cf.  fig.  2); 

•2  •  2  *2 

mr  x  =  x  +  mfcjp  y 

With  ,  _  OB  Ratio  of  the  arms  length  (upper 

OA  and  lower)  of  the  delay  lever. 

So  y  =  p  x 


As  a  matter  of  fact,  In  the  FAMAS  the 
automatic  sear  Is  controled  by  one  of  the  lever 

lower  wings,  replacing  the  present  bolt  unit  by  a 
simple  blowback  lightened  block  meant  that  firing 
was  suppressed  !  As  this  is  precisely  one  of  the 
safeties  of  the  FAMAS.  Moreover,  It  Is  necessary 
that  the  soldier  should  acquire  and  retain  the 
knowledge  of  one  mechanism  for  this  weapon,  and 

this  would  not  have  been  true  any  more  had  the 
working  system  been  transformed  In  the  exchange 
between  the  pieces  of  the  bolt  unit  and  those  of 
the  adapter. 

This  being  said,  and  although  the  working 
principle  of  the  FAMAS  is  well  known.  It  Is  now 

necessary  to  do  some  mechanics  to  understand  the 

whys  of  the  retained  solutions. 

The  FAMAS,  a  small  bullpup  design  works 
according  to  the  principle  of  delayed  blowback  with 
a  delay  lever.  This  lever  couples  the  breech  and 
the  bolt-carrier  (1),  (2),  (3),  (4). 

In  order  to  avoid  the  gripping  of  the  case  in 
the  chamber,  there  are  canelures  along  the  chamber 
In  such  a  way  that  the  case  floats  in  the  gas  with 
an  equal  pressure  on  both  sides  of  the  wall  of  the 
case . 

So,  as  the  pressure  increases,  and  as  the 
bullet  moves  down  the  barrel  and  the  head  of  the 
case  pushes  back  the  breech,  the  bolt  carrier 
starts  to  shift  back,  linked  by  the  delay  lever. 

At  this  time  the  floating  case  acts  as  a 
differential  distributor  of  momentum. 


=  surface  of  the  HEAD  of  the  case 

surface  of  the  MOUTH  of  the  case 


In  fact  p  is  not  a  constant  with  the  FAMAS. 
During  recoil  Its  value  changes  from  P  “  3.6  at 
the  beginning  to  1,  a  mean  value  of  p  «  2.77  Is  a 
good  one  in  most  cases. 

As  a  first  approximation,  these  two  equations 
represent  the  working  of  the  rifle  : 


Pa  =  ~P  Pp 
mr  =  ir>h  -i-  p1  mbc 


For  example  with  the  FAMAS  5.56  :  mr “  3.2  kg 

With  "standard"  ammunition  Pp  B  6  Ns. 

So  for  automatic  firing  the  total  mass  of  the 
moving  parts  should  be  less  than  11.6  kg. 

The  kinetic  energy  of  this  mass  (3.2  kg)  with 
service  ammunition  is  ranging  about  22  Joules*  ; 
and  a  minimum  momentum  of  3  Ns  Is  needed  to  work 
the  FAMAS  with  this  mass. 

To  conclude,  with  a  shortened  range  firing 
system  where  the  cartridge  must  give  out  a  momentum 
ranging  between  1.2  and  1.7  Ns,  it  Is  necessary 
that  the  equivalent  recoiling  mass  should  range 
between  .5  and  1  kg. 


Pg  *  momentum  of  the  rifle  (in  recoil) 


*  1  joule  »  .72  ft. lb. 


372 


The  adapter  of  the  bolt  unit  (cf.  fig.  2  and  3) 

In  eh«  weapon,  the  adapter  oust  substitute 
for  the  following  Items  t 

-  the  bolt  (breech) 

-  the  bolt  carrier 

and  ensure  the  functions  of  the  existing  parts. 

For  human  engineering  (E),  technical  (T)  and 
Industrial  (F)  reasons,  the  following  solutions 
have  been  retained  : 

-  to  keep  the  working  principle  (E  and  T) 

-  to  keep  the  breech  (E,T  and  F)  (nih  =  140g) 

The  only  alterations  effected  In  the  breech 
have  been  performed  for  safety  reasons. 

-  a  smaller  diameter  for  the  recess  In  the 

bolt  head  so  as  to  avoid  the  accidental 
firing  of  a  service  ammunition. 

-  the  setting  of  a  foolproof  device  so  as  to 

forbid  the  Improper  exchange  of  one  of 
the  breeches  for  the  other. 

Then  again,  as  it  was  necessary  to  keep  the 
main  spring  &  miscellaneous  (nyiSO  g)  and  that  the 
design  made  it  quite  Impossible  to  lower  the  ratio 
of  the  lever  beyond  1.6,  it  was  found  that  the 
ammunition  had  to  fire  out  a  momentum  superior  to 
1.3  Ns,  with  a  bolt  carrier  mass  of  about  ISO  g, 
that  Is  to  say  a  total  of  about  300  g  for. 

(mbi  *  m.| ) 


Concerning  the  mass  of  the  rifle  mobile  unit 
elements  and  of  the  adapter,  see  fig.  2,  their 
outline  Is  shown  on  figure  n*  3. 


Fig.  3  -  BOLT  PARTS  (left-Service  items  and 
"balplast"  adapter  at  the  right) 


The  balplast  cartridge 

Since  about  1960  the  French  Army  has  been 
using  with  Its  7.5  armament  (MAS  36/ 51  and  FSA-MAS 
49/56)  an  ammunition.  This  cartridge  only  allows 
hand  operated  firing. 

The  projectile  of  this  cartridge  Is  a  plastic 
ogival  bullet,  made  out  of  orange  colored  RILSAN, 
with  a  flat  brass  base.  This  base  has  a  double 
fonction  as  a  gas-check  inside  the  barrel,  and  as 
a  support  for  the  crimping  of  the  cartridge  mouth. 


At  the  time,  the  characteristics  required  for 
this  projectile  were  as  follows  : 

-  a  7.5  bullet  (.308) 

-  total  weight  of  the  brass  base  :  0.77  g 

-  total  mass  of  the  bullet  :  1.25  g 

At  first  this  cartridge  (see.  fig.  8)  used 
the  service  ammunition  brass  case,  and  this  caused 
many  Incidents.  As  a  matter  of  fact,  the  pressure 
developed  while  firing  being  too  low,  the  case 
could  not  be  an  efficient  gas  seal  In  the  chamber, 
and  this  frequently  caused  discomfort  for  the 
shooter.  In  order  to  put  these  Incidents  right  on 
the  one  hand,  and  to  reduce  the  cost  of  the 
cartridge  on  the  other  hand,  a  much  softer  case 
made  In  a  light  alloy  'as  worked  out,  which  Is 
still  In  service. 


Fig.  8  -  Down  scaling  from  7.5  to  5.56  mm 


The  first  idea  was  therefore  to  reconsider 
this  cartridge  and  adapt  It  to  the  5.56  caliber.  No 
sooner  said  than  done  and  that  was  the  beginning 
of  troubles  all  the  more  embarrassing  as  they 
appeared  late,  too  late.  As  a  matter  of  fact,  these 
cartridges  are  loaded  wtlh  an  extremely  fast 
burning  powder  (the  same  as  the  one  used  in  blank 
cartridges).  As  It  was  not  possible  to  rely  on  the 
crimping  of  the  mouth  of  the  light  alloy  case  for  a 
good  Ignition  of  the  powder,  the  diameter  of  the 
bullet  had  been  sllghly  enlarged  of  0.05  mm 
(.002").  The  forcing  was  sufficient  and  the 
pressure  regular  (on  the  other  side,  this  was  a 
great  contralnt  from  the  industrial  point  of  view 
because  it  meant  recalibrating  the  neck). 

And  then,  one  day.  It  happened  that  the 
impacts  observed  on  a  target  were  tipped  and  even 
keyholed  and  that  dispersion  was  also  aberrant. 
After  a  few  investigations  it  became  obvious  that 
the  profile  of  this  projectile  was  not  suitable 
because  it  was  far  too  sensitive  to  the  barrel 
state  of  wear  (a  phenomenon  that  unknown  In  the 
7.5). 

Under  this  shape ,  it  has  not  been  possible  to 
come  back  to  a  smaller  projectile  diameter  and  to 
ensure  a  better  Ignition  by  using  strong  crimping 
only,  In  particular  by  changing  the  material  of  the 
case  because,  with  a  steel  case,  ruptures  of  the 
projectile  were  observed  to  happen  at  the  upper 
level  of  the  gas  check  due  to  the  crimping. 

Therefore,  the  whole  study  had  to  be  resumed 
from  the  definition  of  the  bullet,  and  this 
resulted  In  the  "FI"  design. 
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Thl*  misadventure  explain*  you  the  presence 
of  two  sketches  on  figure  4,  where  the  diagram  "X" 
Is  related  to  tha  projectile  that  was  left  out. 


It  Is  a  5.71  no  boat-tailed  projectile  ;  Its 
mass  Is  of  0.65  g  (10  grains).  A  real  rotating  band 
constitutes  the  cylindrical  part  upon  which  the 
Rllsan  la  moulded. 


*6* 
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CART  5.56  ( Bullet  V  .  Balded  *fi*  I 


At  present,  works  are  dealing  with  this 
magaslne  so  as  to  make  It  really  polyvalent  with 
ALL  the  cartridges  which  ara  used  (service 
ammunition,  blank,  dummy,  a.s.o.) 


PERFORMANCES  OBTAINED 


The  results  which  are  presented  are  relative 
to  the  works  state  of  advancement  In  Hay  1981 . 

With  the  objective  of  a  presentation  to  the 
technical  official  services  by  the  end  of  the  year, 
the  results  of  these  works  evolve  every  day,  and 
will  not  be  definitely  fixed  until  December  1981. 

This  being  said,  the  nature  of  some  of  the 
tests  may  surprise,  but  one  should  keep  In  mind 
that  this  Is  about  a  pratlce  system  uhere,  up  to  a 
certain  extent,  men  and  projectiles  will  have  to 
coexist  with  somme  reservations  (It  has  not 
happened  so  far,  but  this  projectile  can  cause  an 
almost  fatal  wound  up  to  60  m  and  a  certainly  very 
severe  one  up  to  100  m) . 


It  Is  obvious  that  this  design  is  very  much 
111  suited  to  create  an  Important  drag.  However , 
after  many  tests,  It  Is  the  only  one  which  la 
satisfactory  with  the  arms  on  the  field,  and  which 
enables  us  to  reach  the  level  of  performance 
required  by  the  French  H.Q. 

The  CASE 


Interior  ballistics 

At  the  end  of  this  stage  of  the  study,  it 
should  be  possible  to  choose  a  powder  and  a  charge, 
so  as  to  get  In  a  way  the  required  performances  : 

-  a  momentum  sufficient  to  allow 
automatic  firing 


The  service  ammunition  brass  case  has  been 
chosen,  since,  as  explained  before,  the  steel  one 
was  not  suitable. 

In  the  FAMAS,  there  Is  no  tightness  problem 
since,  on  the  contrary,  the  chamber  Is  designed  so 
as  to  allow  the  powder  gas  pressure  to  release  the 
case  thanks  to  longitudinal  grooves  cut  a  long  the 
wall  of  the  chamber  (the  cartrlges  fired  with  a 
FAMAS  are  easy  to  recognize  by  the  fluting  on  the 
neck) . 


-  a  maximum  pressure  compatible  with  the 
mechanical  strength  of  the  projectile 
for  all  temperatures  ranging  from  -  15*C 
to  +  40*C  (5*  F  to  104*  F) 

-  taking  Into  account  the  small  mass  of 
the  projectile,  the  chosen  powder  had  to 
be  very  fast  burning,  with  a  low  density 
so  as  to  have  a  good  loading  density  of 
the  case ,  and  therefore  good  pressure 
steadiness. 


For  obvious  safety  reasons,  It  is  necessary 
that  It  should  be  impossible  to  fire  service 
ammunition  In  a  rifle  which  has  been  modified  for 
practice  firing.  Apart  from  the  fact  that  the 
components  are  easily  recognizable  by  sight  as  well 
as  by  touch,  It  has  been  Judged  useful  and  safer  to 
reduce  the  diameter  of  the  recess  In  the  bolt  head. 
In  such  conditions,  it  is  not  possible  any  more  to 
fire  unperposedly  a  ball  cartridge. 

This  arrangement  has  brought  about  two  other 
modifications  concerning  : 

-  the  cartridge  case  which  ressembles  now  a 
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-  the  lips  of  the  clip,  which  have  been 
stretched  so  as  to  be  able  to  grasp  the  cartridge 
tightly  while  remaining  compatible  with  the 
rifle's  other  accessories  (loader,  bandoleer...). 

On  the  other  hand,  it  has  been  necessary  to 
study  a  new  magaslne,  the  clips  of  which  are  better 
suited  to  lead  these  rebated  rimmed  cases. 


The  double  base  powders  have  been  eliminated 
because  they  proved  to  be  far  too  erosive.  Within 
400  or  500  rounds,  the  beginning  of  the  rifling  Is 
almost  erased  ! 

In  the  scope  of  this  study,  the  results 
obtained  with  various  temperatures  for  the  same 
powder  have  been  tabulated  hereafter  (table  I). 

In  table  II  the  temperature  is  constant  but 
with  a  different  powder  In  each  case.  The  charge  Is 
the  lowest  possible  providing  reliable  automatic 
functlonning,  with  less  than  1  per  cent  Jamming 
Incidents.  In  fact,  the  charge  from  one  powder  to 
another  varies  very  little,  about  one  grain 
differences  (.065  g). 

Conditions  i  -  Pressure  gun  with  electronic 
transducer 

-  average  firing  ?  three  aeries 
of  ten  rounds 

-  bullet  :.65  g  boattall  FI 

-  brass  cases. 


374 


TAIi.ll 


«• 

v» 

Pm 

<V 

TO 

°r 

• 

«u 

32 

1034 

M 

IAS 

0j07 

3 

+  *1/70 

•10 

11 

•00 

40 

1 » 

0.0S 

u 

.in' 

770 

>7 

see 

•0 

1.1 

0,07 

3 

«°  “  tmupwaWi*  ■*  Clf 

Vw  “  Mtoigjty  it  a  m  from  muuk  (m/i) 
a  («,  p,  t)  “  imndird  dotation 
Pm  “  pramw*  Wwtroele  md  bar) 

tb  “  Imrril  tint*  (rn/an) 

S^y  i ■(®»l  +  °J) 


In  both  tables,  accuracy  is  quoted  for 
Information  only.  However  it  can  be  derived  from 
the  values  mentioned  in  table  I  that  accuracy 
remains  satisfactory  in  the  whole  range  of 
temperatures,  it  can  hence  be  concluded  that  bullet 
material  and  design  are  suitable. 
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EXTERIOR  BALLISTICS 

Ballistic  tables 

Traditionally,  the  designing  work  for  a  new 
cartridge  always  results  in  a  great  number  of  test 
data  being  collected  in  two  large  families  : 

on  the  one  hand  interior  ballistics, 

on  the  other  hand  exterior  ballistics,  the 
latter  resulting  eventually  in  a  ballistic  table. 

The  case  of  terminal  ballistics  in  definitely 
beside  the  question  for  the  moment. 

The  Interior  ballistics  data  could  be 
gathered  with  little  difficulty. 

Reversely,  it  has  not  been  so  far  possible  to 
establish  complete  ballistic  tables  beyond  200  m's 
range. 

Actually,  beyond  this  distance,  the 
trajectory  becomes  so  randomly  that  it  is 
practically  Impossible  to  display  the  measurement 
facilities. 


Let  us  quote  two  observations  in  support  of 
this  assessment! 

at  100  m's  range,  the  accuracy  of  such 
bullets  is  comparable  to,  and  maybe  better  than 
that  of  ball  ammunitions. 


m3.  VELOCITY  V  DISTANCE  iSmvim  MI^'W'| 


Reversely,  at  300  m's  range,  whereas 
accuracies  of  10  cm  s  15  cm  are  frequent  for 
service  ammunition,  out  of  10  balplast  fired  at  a  6 
by  6  m  target  standing  at  the  same  range,  3  bullets 
only  impinged  the  cardboard. 

Again,  during  the  maximum  range  evaluation 
tests,  the  proving  ground  had  to  be  chequered  on  a 
large  area  to  help  recover  the  bullets  on  the 
ground.  In  particular,  3  m/s  gusts  of  wind  may 
alter  the  range  by  over  100  m,  which  is  easily 
explained  by  the  low  weight  of  the  projectile, 
which  has  approximately  the  same  bulk  as  the  M 
193. 


FIS:  7  RANGE 

This  summer,  (198.1)  an  extensive  testing 
program  is  scheduled  for  determination  of  the 
velocity  vs  range  data  as  far  as  the  flight  can  be 
detected,  this  being  capital  information  for 
reliable  determination  of  safety  area. 

Last  year  during  trials  with  a  view  to 
determine  the  ballistic  table  of  a  5.56  bullet, 
doppler  radar  (type  vezero-graph  lambda  10)  was 
used  to  follow  the  trajectory  of  the  projectile  up 
to  900  m's  range. 


375 


Today  thla  facility  aeana  to  ba  our  only  hope 
to  try  and  determine  the  velocity  of  balplast  at 
the  end  of  flight,  without  having  to  close  tha 
horlton  with  acrea  of  cardboard. 

The  results  obtained  up  to  now  are  gathered 
In  table  III  hereafter  and  are  also  to  ba  found  in 
figures  5  and  7. 


TABLI  III 
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2S 
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B0 
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34 
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200 
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10 

39 
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X? 

180 

21.8  Cl 

100 

I  */  On  tho  botit  of  01.  JOURNIB't  work 1 17)  It  it  contidorod  in 
Fnnco  thot  tho  limit  of  brulm  tnd  woundt  In  th « to  ft  pom  of  tnokod  mon 
it  roochod  who n  tho  ortorgy  dontity  of  tho  bullot  it  31  J/cm 2 . 


Considering  this  value,  the  velocity 
threshold  should  be  about  130  m/s  (  425  ft/sec), 

and  hence  the  limit  of  the  hazardous  area  should  be 
about  250  to  300  m  (1,000  ft),  which  confirms  the 
result  of  the  safety  area  determination  led  In  1964 
for  the  Mle  61  bullet  of  the  7.5  mm  cartridge. 

In  order  to  give  a  more  precise  Idea  of  the 
magnitude  of  this  energy  density 

•Ib/sq.ln.),  this  corresponds  approximately 
to  the  energy  of  a  No  7  ^  pellet  70  yards  from  the 
shooter. 

NOTA  ;  For  this  and  the  following  paragraphs,  all 
data  quoted  refer  to  firings  performed  with 
rifles  equipped  with  a  balplast  kit  ;  the 

the  production  line,  and  have  fired  about 
1,500  rounds  In  previous  tests.  All  firings 
are  performed  by  able  shooters  (not 
marksmen),  using  the  production  type 

bipod . 

Accuracy 

On  firing  balplast  cartridges  with  the  rifle, 
a  displacement  of  the  mean  point  of  impact  Is 
observed  with  respect  to  the  results  obtained  with 
ball  ammunition. 

Thla  displacement  results  In  a  target  at 
lOOm’s  range  by  :  (cf.  fig.  9  hereafter) 

3  to  5  cm  drift  to  the  right 

15  cm  rise  In  elevation 

It  Is  most  likely  that  nothing  can  be  done  sb 
regards  the  drift,  the  latter  being  inherent  to 
flight  physics  ;  on  the  other  hand.  It  has  been 
observed  that  by  adding  a  muzzle  mounted  device,  it 
was  possible  to  decrease,  and  even  suppress  the 
rise  In  elevation.  Works  are  being  led  In  that 
line. 


Fig.  9  Shifting  of  the  mean  point 

Table  IV  Is  a  report  of  the  reaulta  obtained 
on  rifle  firing  In  the  conditions  below  : 

•Moer  ram* 

■  flrtnt  on  tM  bench  with  biped  at  ( ten 

•  metallic  dfhm 

■  rHIt  Nr  P127  equipped  with  Balplan  adapter 

•  10  round!  Mr  let  find  thot  by  drat 

■  BS 10  Ba  powder 


TABLE  IV 
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NOTA  :  Unloot  othorwim  montionod,  oil  mooturot  on  in  cm. 


Time  Of  flight 

The  system  being  designed  for  elementary  training 
and  practice  to  automatic  firing,  Its  utilization 
should  in  no  case  develop  In  the  shooter  reflex 
behaviour  differing  in  any  way  from  that  implied  by 
firing  service  ammunition. 

We  have  seen  above  what  concerns  accuracy  (group 
and  zero).  For  time  of  flight,  all  data  obtained 
are  tabulated  versus  time  of  flight  of  the  service 
ammunition  in  fig.  6. 

At  100  m's  range,  the  time  of  flight  of  balplast  Is 
0.05  sec  longer  than  that  of  the  ball  ammunition, 
which  would  result  In  a  necessity  for  25  cm 
correction  in  the  case  of  a  target  moving  sideways 
at  5  m  per  sec  velocity  (which  Is  a  maximum). 

In  the  same  conditions,  between  ZERO  and  70  m, 
target  correction  is  less  than  10  cm  (plus  or 
minus) . 

Such  values  should  prove  a  hindrance  for  the 
probability  of  hit,  «s  well  in  shot  by  shot  firing 
as  burst  firing. 
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EUL4  TIME  -QF .  FLIGT 

SAFETY 


b)  beyond  the  above  mentioned  Halt  angles, 
the  projectile  more  or  less  deformed,  will  skid 
along  the  surface.  Use  of  metallic  plates  for 
protection  Is  therefore  not  advisable  if  grazing 
fire  is  likely  to  be  perforaed. 


Foolish  interferences 


Experience  unhappily  proves  that  every  year, 
in  spite  of  all  advice,  soae  unconscious  people 
(the  tern  is  euphemistic)  will  meddle  with 
cartridge  modification,  and  get  spectacular 
ballistic  results  endangering  not  only  their  own 
lives  but  also  their  neighbours'. 

It  is  worth  knowing  what  will  happen  if  a 
service  ball  is  mounted  instead  of  the  balplaat. 
Considering  the  type  of  powder  used  (BS  30,  hence 
fast  burning),  we  may  wonder  which  end  the  whole 
stuff  is  to  be  expected  to  pop  off...  We  plan  to 
film  the  experiment  for  training  purpose. 


The  maximum  range  is  about  500  to  700  m, 
according  to  wind  velocity,  but  at  such  distances, 
the  danger  is  null. 


300  m  is  likely  to  be  che  limit  retained  for 
the  hazardous  area  (according  to  the  criteria  taken  CONCLUSIONS 

in  consideration) . 


The  system  being  liable  to  be  used  in  firing 
course  practice,  the  necessary  protections  and  the 
behaviour  of  the  projectile  at  short  range  had  to 
be  determined  i 

At  A 5  m  (50  y)  and  for  a  remaining  velocity 
of  800  m/sec  (2,620  ft/sec)  under  zero  incidence, 
the  minimum  thickness  of  protective  material  is  : 

-  100  mm  for  poplar  wood 

6  mm  for  2017  A  light  alloy  ( AU4G) 

3  mm  for  A  33  mild  steel 

50  Z  penetration  of  41  mm  thick  poplar  wood 
is  obtained  at  600  m/sec  velocity,  that  is  at 
65  m's  range  (71  yards) 


This  communication  is  the  syntheses  of  works 


led  by  the  research 

teams  in 

4  arsenals  of  GIAT 

Pyrotechnics  and  cartridges 

A.L.M. 

Atelier  de  fabrication  du  Mans 

Pyrotechnics  and  cartridges 

A.T.S. 

Atelier  de  cunttnrction 
de  T  ARSES 

Rifle 

M.A.S. 

Manufacture  Ns tiona is 
d'Armtt  da  SAINT-ETIENNE 

Project  management 

E.F.A.B. 

Etabitaamant  d'Etudst  at  da 
Fabrications  d'Armamant 
ds  BOURGES 

Rebounce 


Still  in  the  scope  of  firing  course  practice 
and  more  precisely  for  street  fighting,  or  cine 
simulator  the  behavioux  of  the  balplast  on  impact 
against  various  materials  was  observed  : 

a)  in  all  cases  there  is  no  rebounce  or 
fragment  projection  beyond  7  m  above  the  limit 
angles  of  impact  given  hersafcer  : 


-  mild  'teel  10' 

-  light  alloy  30° 

-  concrete  60“ 

-  wood  80“ 


As  concerns  wood,  the  projectile  sticks  deep 
into  the  protector  and  does  not  get  through,  even 
should  5  or  6  projectiles  stack  up  in  the  same 
hole.  For  other  materials  the  projectile  bleaks  up 
into  unsignificant  fragments  with  less  than  7  m 
lateral  range. 


It  reports  the  present  stage  of  development 
in  May  1981  of  the  "shortened  range  practice 
system"  adapted  to  the  French  assault  rifle. 

The  system  definitely  maintains  the  same 
human  engineering  and  performance  criteria  of  the 
service  weapon,  and  provides  the  same  types  of 
firing  modes  as  with  the  ball  ammunition. 

In  particular,  the  system  will  allow  training 
of  the  combat  group  to  instinctive  automatic  firing 
in  adapted  firing  courses  with  a  hazardous  area 
limited  to  300  m. 

There  still  remains  much  to  be  done  before  final 
homologation,  yet  the  characteristics  of  the  system 
are  already  definetcly  settled  and  production 
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SURVIVING  THE  riTFAI.LS  AND  TROLL  BRIDGES 


IN  STATE-OF-THE-ART  CBI  DEVELOPMENT 
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ABSTRACT 

Most  computer-based  instructional  (CBI)  system  development  is  done  using  proven  technologies 
in  proven  way9.  Sophisticated  development  models  are  available  for  determining  software  (e.g., 
top-down  design)  and  courseware  (ISD)  decisions  and  structures  in  these  systems.  For  state-of- 
the-art  (SOA)  CBI  system  development,  however,  there  are  additional  probelms  peculiar  to  that 
form  which  require  specific  considerations  and  solutions.  This  paper  is  designed  to  define  the 
specific  characteristics  of  thi3  type  of  development  and  to  discuss  some  of  the  problems,  con¬ 
siderations,  and  solutions  involved  in  SOA  training  system  research  and  development.  These 
Include  personnel  concerns,  such  as  work  environment  characteristics  and  the  type  of  people  best 
suited  to  it;  management  concerns,  such  as  the  style  of  management  required;  communications 
concerns,  such  as  the  required  interaction  between  different  staff  disciplines;  and  the 
different  technical  approaches  required  for  this  specific  form  of  development. 


INTRODUCTION 


Logicon  specializes  in  the  integrated  appli¬ 
cations  of  advanced  technologies  and  advanced 
concepts  to  system  engineering  problems.  Within 
this  context,  the  Tactical  and  Training  Systems 
Division  has  undertaken  several  state-of-the-art 
CBI  (computer-based  Instruction)  system  development 
projects.  (1,2,3)  The  purpose  of  this  paper 
is  to  present  some  of  the  problems  associated 
with  doing  work  in  tills  area  and  some  possible 
methods  for  avoiding  those  problems. 

The  problems  covered  in  this  paper  are  of 
basically  two  types:  pitfalls  and  troll  bridges. 
Pitfalls  are  those  problems  which  occur  where 
you  don't  expect  them.  You  think  you're  on  solid 
ground  and  then  suddenly  you're  at  the  bottom 
of  a  hole  with  a  lot  of  angry  natives  looking 
down  at  you.  Troll  bridges  are  generally  problems 
which  you  know  about.  The  trolls  have  a  unique 
way  of  conducting  business:  they  peacefully 
acoept  whatever  payment  you  believe  is  reasonable 
for  crossing  the  bridge.  Then,  when  you're  at 
the  middle,  they  demand  you  pay  more  or  they 
toss  you  off. 

It  is  important,  at  this  point,  to  define 
some  terms.  SOA  development  means  attempting 
to  implement  the  most  recent  technologies  available 
within  a  given  field.  This  is,  by  definition, 
a  high  risk  endeavor  with  unknown  coma^dities. 
It  makes  the  developer  vulnerable  to  many  levels 
of  non-3uccess.  A  CBI  system  is  based  upon  oomputers 
and  associated  peripheral  devices  such  as  CRTs, 
printers,  audiovisual  delivery  systems,  and  compu¬ 
terized  speech  generation  and  voice  recognition 
hardware.  The  techniques  used  include  computer 
assisted  instruction  (CAI),  ocmputer-based  performance 
measurement  (PM),  and  computer  managed  instruction 
(CMI) . 


In  an  area  as  diverse  as  state-of-the-art 
(SOA)  development  of  CBI,  there  are  lots  of  pitfalls 
and  troll  bridges.  They  come  in  large  (general) 
and  small  (speciflo)  sizes  and  can  appear  anywhere. 
This  paper  attempts  to  present  some  of  the  ones 
of  both  sizes  that  Logicon  has  encountered  in 
several  different  development  areas. 


This  paper  has  been  structured  to  move, 
more  or  less,  from  general  to  specific.  The 
first  three  sections  deal  with  mere  general  concerns. 
These  sections  are  titled  "State-of-the-Art,  " 
"Computer-Based  Instruction,"  and  "People." 
The  final  two  sections  delve  into  more  of  the 
specifics  of  CBI  development.  These  sections 
are  titled  "New  Hardware  to  USE"  and  "New  Things 
We're  Doing." 


STATE-OF-THE-ART 

There  are  generally  two  sets  of  attitudes 
regarding  SOA  technologies.  These  can  be  categorized 
as  "bandwagon"  and  "tradition-based."  A  good 
example  of  the  bandwagon  approach  is  the  present 
"love  affair"  going  on  with  the  videodisc.  It 
seems  that  lately  Just  about  every  other  training 
system  RFP  has  requested  the  use  of  videodisc. 
In  many  cases,  however,  traditional  technologies 
could  provide  better  performance  with  less  risk. 

The  tradition-based  approaoh  is  characterized 
by  people  not  using  new  technologies  solely  because 
the  technologies  are  new  or  using  them  as  an 
expensive  version  of  the  more  traditional  media. 
An  example  is  the  use  of  videotape  in  the  classroom. 
Initially,  there  was  a  great  deal  of  resistance 
from  teachers  because  they  thought  they  were 
going  to  be  replaced.  Then,  when  television 
finally, was  accepted  into  the  classroom,  if  was 
often  used  to  present  lectures. .. Just  as  though 
there  was  a  teacher  standing  there. 

Because  there  is  no  body  of  experience  upon 
which  to  base  design  and  development  decisions, 
it,  is  important  to  realize  that  state-of-the- 
art  development  Involves  taking  risks  and  makes 
you  vulnerable  to  failure.  Thus,  there  must 
be  a  oonmitment  to  this  vulnerability.  RAdm.  Albert 
J.  Baclocco  provides  this  perspective: 

"When  I  state  a  willingness  to  take  risks, 
I  am  simultaneously  stating  a  willingness 
to  accept  some  failures.  In  my  view,  failure 
is  also  a  measure  of  success  in  basic  re¬ 
search. ..as  long  as  it's  not  100*  failure." 
(4) 
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If  you  get  Involved  in  producing  a  "suc¬ 
cessful  product"  the  first  time  out,  you'll  most 
likely  limit  your  use  of  the  technologies  to 
the  more  familiar  realms  rather  than  exploring 
the  further  limits  of  their  capacities.  If  the 
decision  has  been  made  to  do  SOA  development, 
it  is  important  to  apply  the  knowledge  that  has 
been  accrued  from  work  in  similar  areas.  This 
knowledge  should  be  used  as  a  starting  point 
for  identifying  means  for  using  the  SOA  technologies 
in  way  most  amenable  to  their  inherent  character¬ 
istics.  If  there  is  no  commitment  to  vulnerability 
in  the  use  of  the  new  technology,  you  would  probably 
be  better  off  adapting  an  older,  more  familiar 
and  secure  approach. 

Early  in  the  project  a  needs  assessment 
should  be  done.  In  this  case,  you  are  assessing 
your  need  to  actually  utilize  new  technologies: 
how  are  they  appropriate,  what  do  they  add,  and 
are  they  cost  effective.  In  some  oases,  you 
may  be  responsible  for  providing  a  technology 
test  bed.  In  that  case,  you  should  carefully 
examine  the  possible  applications  for  ones  that 
will  stretch  the  limits  of  the  technology.  When 
you're  not  responsible  for  testing  the  technologies, 
you  should  carefully  analyze  your  application 
and  choose  the  technologies  (be  they  new  or  be 
they  old)  that  most  closely  fit  your  identified 
needs,  resources,  and  allowable  risks. 


COMPUTER-BASED  INSTRUCTION 

In  CBI,  as  with  most  instructional  approaches, 
it  is  important  to  define  a  task  that  can  be 
accomplished  with  your  given  resources  (people, 
time,  money).  The  task  itself  should  be  driven 
by  considerations  of  the  instructional  system 
outcomes:  what  is  the  system  supposed  to  do, 
how  quickly,  for  how  many  people,  how  well. 

In  most  instructional  approaches  the  instruc¬ 
tional  designer  knows  the  medium  sufficiently 
well  to  make  many  intelligent  decisions  concerning 
equipment,  budget,  and  approach.  CBI,  because 
it  involves  complex  and  expensive  components, 
requires  three  sets  of  specialists  who  auat  be 
involved  in  most  of  the  initial  system  decisions 
and  who  must  communicate  continuously  during 
the  design  and  development  process  to  insure 
that  everyone  understands  what,  precisely,  is 
going  on. 

The  three  sets  of  specialists  are  hardware, 
software,  and  courseware  designers  and  developers. 
Since  they  represent  entirely  different  parts 
of  the  system  development  process,  communication 
to  ensure  understanding  becomes  imperative. 
These  people  must  establish  a  mutually  held  concept 
of  the  system  and  its  capabilities  before  the 
limiting,  defining,  and  restricting  decisions 
(e.g.,  what  hardware,  what  software  language, 
what  instructional  outcomes)  have  been  made. 
Then  they  must  continue  the  cross  discipline 
communication  to  ensure  that  the  picture  stays 
congruent  for  all  three  staff  disciplines. 


PEOPLE 


There  are  several  sets  of  people  involved 
in  the  development  process  of  SOA  CBI.  In  addition 
to  staff,  there  are  managers  and  users.  The 
users  can  be  further  categorized  into  students 
and  instructors.  This  seotion  addresses  pitfalls 
and  troll  bridges  associated  with  eaoh  of  these 
sets  of  people. 

Managera 

SOA  CBI  development  often  laoks  muoh  of 
the  extrinsic  structure  oommon  to  training  systems 
which  use  more  traditional  methods  and  media. 
Additionally,  the  stress  levels  will  typioally 
be  higher  as  a  reflection  of  the  risk  (chance 
for  error)  associated  with  exploring  new  territories. 

Many  of  the  problems  associated  with  this 
area  may  be  avoided  by  maintaining  a  more  active, 
more  psychological  project  management  approach 
than  usual.  It  is  important  to  reduce  staff 
stress.  This  may  be  done  by  supplying  strong 
leadership  to  provide  structure  where  it  is  absent. 
Managers  will  want  to  encourage  innovation  and 
analysis  at  the  outset  to  insure  that  the  technology 
is  being  utilized  in  the  best  possible  way. 
They  will  want  to  ensure  communication  between 
the  groups  (hardware,  software,  courseware). 

Perhaps  most  Importantly,  managers  should 
identify  "inch  pebbles"  (as  opposed  to  milestones) 
for  identifying  project  progress  and  should  be 
very  aware  of  places  where  the  inch  pebbles  are 
not  being  reached.  This  will  provide  a  methodology 
for  forestalling  the  large  crises  to  which  SOA 
development  is  more  vulnerable. 

Staff 

In  attempting  SOA  CBI  development,  your 
choice  of  staff  members  becomes  a  little  more 
important.  Of  course  you  will  want  to  choose 
people  with  as  good  skills  and  experience  as 
are  available  to  you.  However,  staff  members 
for  SOA  development  should.  If  possible,  be  flexible 
persons  with  low  "final  product"  attachments. 
SOA  work  does  not  lend  itself  to  obvious  first 
time  success  and  often  requires  multiple  iterations 
or  complete  redesign  in  order  to  derive  a  useful 
product.  Indeed,  sometimes  you  may  find  that 
success  with  a  particular  technology  configuration 
is  impossible.  Therefore,  a  person  working  on 
SOA  CBI  would  be  better  served  to  get  satisfaction 
from  getting  a  job  done  as  well  as  they  can, 
rather  than  depending  on  traditionally  successful 
end  results. 


Students 

In  terms  of  the  student-system  interface, 
CBI  presents  the  design/development  team  with 
a  new  set  of  considerations.  Major  amongst  these 
considerations  are  the  learners'  attitude  toward 
automated  instruction,  how  to  supervise  Information 
presentation,  and  how  to  teach  the  learner  to 
use  the  system. 

Learner  Attitudes.  There  are  two  main  attitude 
p.obleos.  Up  until  the  recent  past  the  average 
person  has  stood  in  awe  of  computers.  Learners 
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have  been  initially  retieent  to  use  conputer 
systems  for  fear  cf  "breaking*  the  very  expensive 
equipment.  With  the  rapid  lnolusion  of  computers 
into  many  aspects  of  day-to-day  life  this  oonoern 
i 3  diminishing,  but  still  must  be  considered. 
Fears  have  also  been  voioed,  on  a  more  or  less 
continuous  basis,  about  computers  dehumanizing 
the  learning  experience. 

The  designer  must  take  both  these  factors 
into  consideration.  The  OBI  system  must  be  user 
oriented.  It  is  wise  to  go  to  the  learner  population 
and  poll  their  attitudes  about  computers.  By 
concentrating  on  Identifying  any  fears  the  students 
have  about  CBI,  you  can  know  where  to  start  in 
providing  a  stress  reducing  Introduction  to  the 
system  ana  the  experience.  Both  the  computer 
fears  and  the  possibility  that  the  lack  of  human 
Intervention  will  degrade  the  instructional  effec¬ 
tiveness  can  be  at  least  partially  solved  through 
(1)  careful  inclusion  of  humans  into  the  system 
design  and  (2)  giving  the  computer  a  "person¬ 
ality."  Building  a  human-system  rapport  can 
serve  to  reduce  both  the  potential  effects  of 
dehumanization  and  learner  fears  about  system 
use. 

Presenting  Information.  A  compvter-based 
Instructional  system  can  provide  tne  learner 
with  information  in  a  variety  of  ways.  These 
include  CRT,  various  audiovisual  media,  printers, 
and  computer  generated  speech.  Guidelines  about 
how  to  present  information  using  various  media 
already  exist. 

(5)  is  one  good  example. 

However,  when  you  have  mere  than  one  information 
source  available  in  your  CBI  system,  your  problem 
becomes  one  of  directing  the  learners'  attention 
from  one  source  to  the  next  and  ensuring  that 
the  learners  know  precisely  what  behavior  is 
expected  of  them  at  all  times.  Not  knowing 
how  to  initiate  the  next  step  can  cause  the  learner 
to  experience  a  sort  of  stress-induced  paralysis 
or  may  result  in  a  random  pushing  of  keys  and 
buttons  until  won  thing  happens. 

Learning  How  Use  the  System.  The  use 
of  computer-based  systems  provides  you  with  another 
new  problem,  that  of  familiarity.  The  student 
will  not  usually  be  familiar  with  computerized 
instruction  in  general  and  your  system  in  particular, 
lou,  therefore  are  left  with  the  task  of  teaching 
the  student  how  to  use  the  teaching  system. 

This  is  a  very  important  problem  inasmuch 
as,  if  the  student  cannot  use  the  system,  the 
system  cannot  train.  It  is  critical  that  you 
identify  what  the  student  will  need  to  do  to 
use  the  system  and  then  carefully  teach  those 
behaviors.  It  is  a  good  idea  to  separate  this 
instruction  from  the  general  instruction  so  the 
student  may  review  aspects  of  system  use  whenever 
he  is  unsure  of  what  he  is  supposed  to  be  doing 
to  make  things  go. 

Instructors 

As  with  the  students,  the  instructors  are 
users  of  this  system.  Their  responsibilities 
will  usually  extend  to  managing  system  use  and 
providing  a  human  control  over  system  use  and 
misuse  by  the  students.  Their  attitudes  and 
knowledges  must  be  considered  in  your  system 
design  and  development. 


Instructors  will  generally  resist  the  imple¬ 
mentation  of  a  CBI  system  within  their  teaching 
realm.  This  is  a  result  of  fears  that  they  are 
going  to  be  replaced  by  the  system  or  that  they 
will  become  simple  technician  adjuncts  while 
the  system  does  the  Important  tasks.  In  our 
experience  with  CBI,  the  instructors  generally 
beoome  comfortable  only  when  they  see  they  have 
some  control  over  the  system  (6)  and  that  they 
are  still  Involved  in  important  Instructional 
system  tasks. 

It  becomes  very  important,  therefore,  to 
make  the  system  as  easy  to  supervise  as  possible. 
It  also  beoomes  very  Important  to  provide  the 
instructors  with  training  on  the  roles  that  they 
are  expected  to  fulfill  and  to  accentuate  how 
important  these  roles  are  to  system  success. 
In  one  system  presently  under  development,  we 
are  developing  an  instructor  tutorial  using  the 
system  to  teach  the  instructors  as  well  as  the 
students. 

In  any  case,  the  current  SOA  CBI  system 
tends  to  put  the  Instructor  into  three  roles: 
system  manager  (controlling  and  logging  system 
operation),  Instructional  facilitator  (overseeing 
and  smoothing  system-student  Interactions),  and 
tutor.  Since  most  instructors  are  only  familiar 
with  being  in  the  role  of  primary  instructional 
resource,  the  three  new  roles  must  also  be  taught. 


NEW  HARDWARE  TO  USE 

The  new  hardware  which  is  mentioned  below 
is  but  a  small  sample  of  the  wondrous  things 
which  are  becoming  available  for  use  in  instructional 
systems.  In  this  paper  the  two  types  of  hardware 
discussed  are  automated  speech  hardware  and  the 
videodisc. 

AuLsofttf  A.  £p»£fih.  Mar.dw.arfi 

Automated  speech  includes  speech  generation, 
speech  recognition,  and  speech  understanding. 
Speech  generation  and  recognition  are  mostly 
hardware  based  and  will  be  discussed  in  this 
section.  Speech  understanding  is  mostly  software 
based  and  will  be  discussed  in  the  NEW  THINGS 
WE'RE  DOING  section. 

AuLfiBJLt.fi  d .  &B4M CL. -0.6 DfiJlfti Ian .  There  are 
a  variety  of  devices  available  which  can  produce 
speech  under  computer  control.  Most  of  these 
devices  fall  into  two  general  categories:  speech 
synthesizers  and  speech  encoder/decoders. 

Speech  Synthesizers.  The  synthesizers  produce 
artificial  sounding  but  intelligible  speech  by 
combining  electronically  generated  sounds  which 
correspond  roughly  to  English  phonemes.  Some 
synthesizers  are  capable  of  performing  the  translation 
from  English  text  to  speech  directly;  others 
require  human  intervention  to  specify  the  phoneme 
strings  and  inflections.  The  devices  also  vary 
in  cost  and  in  the  intelligibility  of  the  speech 
output.  The  synthetic  quality  of  the  speech 
has  been  identified  as  a  disadvantage,  but  the 
small  data  storage  and  I/O  processing  overhead 
requirements  make  synthesizers  an  attractive 
solution  to  many  speech  generation  problems. 
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The  one  outstanding  drawback  to  the  synthesizer 
enoountered  in  certain  applications  is  the  fat  t 
that  the  simulation  of  many  different  voloes 
is  impossible.  Another  difficulty  is  that  it 
is  usually  impossible  to  vary  speech  rate  under 
program  control.  Even  at  the  highest  available 
speeoh  rate,  the  synthesizers  tend  to  speak  relatively 
slowly.  This  oaused  difficulties  when  they  were 
used  to  simulate  air  traffic  controllers  because 
the  synthesizer  was  sometimes  unable  to  provide 
enough  control  information  in  the  time  allotted. 

Speech  Enooder/Decoders.  These  devioes  utilize 
a  technique  which  digitally  encodes  and  compresses 
of  human  speech  in  suoh  a  way  that  it  oiin  be 
subsequently  decoded  to  produoe  speeoh.  These 
devices  provide  a  range  of  fidelity  from  superb 
(and  expensive)  audiophile  quality,  to  lower 
cost  units  whioh  provide  oorrespondingly  lower 
fidelity.  Even  the  low  fidelity  units  produce 
speech  which  is  unmistakably  human  and  sufficien¬ 
tly  intelligible  for  many  applications.  These 
devices  can  easily  be  used  to  simulate  different 
voices  simply  by  recording  voices  of  different 
talkers.  Depending  upon  the  encoding  technique 
used,  these  devices  oan  offer  the  advantage  that 
speech  is  simply  recorded,  and  there  is  no  need 
for  painstaking  encoding  of  phoneme  strings. 
Others  require  off-line  processing  of  the  encoded 
Information  to  reduce  the  data  requirements  to 
practical  levels.  The  disadvantage  of  the  enooder/ 
decoders  lies  in  their  relatively  much  larger 
data  storage  and  I/O  processing  requirements. 

Speech  Recognition.  Commercially  available 
hardware  ranges  from  inexpensive  units  which 
recognize  a  few  words  spoken  in  Isolation  to 
more  expensive  devices  whioh  recognize  a  large 
vocabulary  of  isolated  words  or  phrases  with 
high  accuracy,  to  even  more  expensive  devices 
which  recognize  continuous  speech.  Most  units 
are  still  "speaker  dependent"  which  means  they 
must  be  configured  to  recognize  an  individual 
talker's  voice  by  having  the  person  repeat  each 
vocabulary  item  one  to  ten  times. 

Our  experience  has  shown  that  recognition 
accuracy  is  affected  by  several  factors,  Including 
the  vocabulary  itself.  In  some  applications, 
the  vocabulary  is  defined  a  priori.  In  many 
others,  it  can  be  defined  by  the  system  designer. 
In  the  latter  case,  great  care  should  be  devoted 
to  the  development  of  a  vocabulary  tailored  both 
for  user  acceptance  and  for  machine  recognition. 
The  optimization  for  machine  recognition  almost 
has  to  be  done  by  experimentation  because  the 
machine  ear  has  different  characteristics  from 
the  human  ear.  The  inability  of  many  systems 
to  distinguish  reliably  between  "five"  and  "nine" 
is  often  cited.  In  our  applications,  we  have 
also  found  many  other  pairs  which  cause  trouble, 
Including  "above"  and  "below,"  "left"  and  "right," 
and  "port"  and  "four." 

JfldepiUac 

The  videodisc  is  a  marvelous  tool  for  providing 
color,  still,  or  moving  pictures  and  high  quality 
audio  under  computer  control.  It  could,  of  course, 
be  used  to  provide  lecture  material,  but  its 
great  advantage  lies  in  the  fact  that  demonstrations 
by  experts,  mini  reviews  of  speoific  topics, 
etc.,  can  be  randomly  accessed  by  the  computer 


or  at  trainee  request.  Theoretically,  the  technology 
oould  provide  the  basis  for  a  talking,  moving 
book,  with  a  sophisticated  Indexing  and  retrieval 
capability. 

The  disadvantages  to  the  technology  lnolude 
that,  unlike  videotape,  the  production  aspect 
is  akin  to  audio  record  mastering,  henoe  the 
audiovisual  materials  cannot  readily  be  changed. 
The  unreliability  of  early  devices  also  has  proved 
problematical. 

It  is  very  important  to  carefully  Identify 
precisely  what  kinds  of  graphics  and  sound  you 
wish  to  present  in  your  CBI  system  and  then  make 
sure  that  videodisc  is  the  proper  presentation 
medium  for  your  needs.  Once  that's  determined, 
your  next  concern  is  ohoosing  the  videodisc  player 
appropriate  for  your  application.  There  are 
several  varieties  of  this  technology  presently 
available  with  others  undergoing  prototype  develo¬ 
pment. 

Another  important  hurdle  is  production. 
Since  this  is  a  relatively  infant  medium,  many 
of  the  production  bugs  still  must  be  worked  out. 
Premastering,  mastering,  and  quality  assurance 
are  all  areas  to  whl«h  you  should  pay  attention. 
(7)  It  is  also  important  to  note  that,  as  of 
the  beginning  of  this  year,  none  of  the  videodisc 
production  companies  had  the  ability  to  do  classified 
premastering  or  mastering.  If  you  have  security 
related  materials  to  show  you  should  check  carefully 
to  see  if  this  problem  has  been  overcome. 


NEW  THINGS  WE'RE  DOING 


This  section  describes  some  of  the  parts 
of  the  CBI  system  such  as  courseware,  automated 
Instructor  model,  and  automated  speech  under¬ 
standing.  Herein  are  described  some  of  the  problems 
associated  with  working  in  these  SOA  areas. 

Courseware 

In  a  CBI  system  the  courseware  materials 
are  mostly  embedded  In  the  computer  memory. 
Therefore,  the  courseware  language  you  use  becomes 
very  important.  Because  of  the  iterative  nature 
of  SOA  CBI  development,  it  becomes  practically 
a  necessity  to  be  able  to  a  quick,  easy  review 
of  the  materials  you  have  developed.  As  you 
add  hardware  devices  (e.g.,  videodisc,  simulated 
job  station)  or  capabilities  to  your  system, 
it  becomes  increasingly  important  to  have  a  supporting 
courseware  language  which  offers  simplicity  in 
your  specification  of  devices,  what  they  do, 
when  they  do  it,  and  how  long  they  do  it.  The 
language,  if  possible,  should  also  provide  for 
easy  revision  of  materials. 


A  typical  CBI  system  development  will  require 
parallel  development  of  software  and  courseware. 
Unless  you  have  a  very  long  timeline  and  can 
develop  and  test  the  whole  system  before  you 
have  to  write  and  enter-  any  of  your  materials 
into  computer  memory,  it  is  a  good  idea  to  devise 
your  system  so  that  you  can  see  and  revise  pieces 
(l.e.,  text  pages,  scenarios)  before  the  entire 
system  is  finished. 
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There  ere  several  different  courseware  languages 
available  on  different  systems  (e.g.,  Pilot, 
Deoal)  and  others  under  development.  If  none 
of  those  you  can  find  offer  what  you  need,  you 
may  want  to  do  as  we  have  done  and  develop  your 
own  courseware  language  to  precisely  fit  your 
application.  If  you  choose  to  develop  a  language, 
define  very  carefully  amongst  the  three  work 
specialties  (courseware,  software,  and  hardware) 
what  the  Instructional  system  will  be  designed 
to  do,  given  your  particular  resource  parameters. 

Au,tpRB.t.e.d_  Instructor 

The  simulation  of  any  complex  human  behavior 
is  certainly  not.  a  trivial  task.  In  the  course 
of  our  work  In  automating  the  instructor's  task 
of  evaluating  a.  trainee,  wc  have  come  to  appreciate 
Just  how  ncn-trivisl  the  usserrm* rt  of  humsr, 
performance  car  be!  The  following  Oi  sci-sai.ee 
highlights  some  of  the  pitfalls  ar.d  troll  bridges 
we  have  encountered. 

£flanaaftDLs.£JL.M  -  ivtsmteA  JjmirHPipr.  p app.- 

bllltv.  An  automated  instructor  capability  should 
have  several  components.  It  should: 

•  Provide  an  objective  measure  of  trainee 
performance  on  a  particular  problem 
relative  to  objective  criteria; 

•  Collect  data  suitable  for  statistical 
aril) nil-  mul  rerm  development; 

•  Fro vide  the  human  instructor  with  a 
meaningful  assessment  of  trainee  performance 
at  a  particular  stage  of  Instruction, 
as  well  as  an  overall  assessment; 

•  Provide  positive  and  negative  feedback 
and  annotated  replay; 

•  Provide  adaptively  tailored  learning 
materials. 

Distinguishing  all  these  capabi]  ities  maxes 
it  clear  that  the  specification  of  the  raw  perform  .  c 
tlrta  which  the  system  Is  to  collect  requite: 
ntfJKi!  t  m  k  ertlytif,  and  fortltr  that  ».•  .  st. 

data  must  he  anflys;cd  in  various  way;.  «. 
■eve  meaningful  ends. 

faak  Analysis.  The  CBI  system  designer 
:  at  a  great  disadvantage  with  respect  to  other 
training  program  developers.  When  one  has  a 
human  instructor  to  deal  with,  one  can  slmtiy 
say,  "Mach  this."  The  computer  Is  obvioucty 
ureble  to  do  that.  It  can  be  a  profound  cheek 
tc  the  subject  matter  expert  (SHE)  to  find  *.nat 
the  task  analysis  has  to  be  specified  to  an  inored. tly 
(retailed  level  to  satisfy  the  needs  of  the  '.El 
system.  In  fact,  SMEs  are  asked  to  quart  ify 
those  qualitative  judgements  which  they  make 
almost  automatically.  This  can  prove  to  be  very 
difficult.  The  system  designer  must  be  especia-.y 
sensitive  to  the  fact  that  the  SHE  is  being  jsh* 

• ;  think  in  a  new  way  and  that  revisions  '•  :h> 
the  task  analysis  outcomes  will  be  inevitable 

Measures  of  Perf orttancji.  On  the  one  bird, 
atst  1  sti t lane  luve  demonstrated  that  the  overall 
Performance  evaluation  an  expert  instructor  provides 
can  be  emulated  by  considering  only  a  very  small 
number  of  measures  of  performance.  Statistical 
techniques  like  these  are  very  useful  for  the 
sped!  ic  purpose  of  providing  an  overall  grade. 


Howev.r,  it  would  he  a  mistake  to  conclude  that 
other  measures  were  of  no  use.  In  fact,  a  rich 
'f  ritasures  baaed  upor  the  task  analysts 
dr  1 1  If  reeded  to  enable  the  training  system 
to  identify  the  trainee  mistakes  which  leu  p 
to  an  undesirable  outcome  and  provide  indications 
of  correct  procedures. 

Like  any  test  items,  these  performance  measures 
must  be  tested,  validated,  refined,  and  tested 
again.  Whan  this  has  been  done,  it  la  necessary 
t.o  combine  the  measures  to  provide  various  overall 
measures  of  trainee  performance.  Again,  these 
combined  ncanures  must  be  veil  dated  end  refined. 
It  is  foolhardy  to  ignore  this  basic  principle 
of  test  instrument  development  just  because  the 
test  Items  do  not  lock  like  standard  test  items. 


The  validation  procedure  may  reveal  problems 
not  encountered  in  the  validation  of  standard 
items.  For  example,  in  evaluating  ongoing  perfor¬ 
mance,  there  can  be  a  "domino  effect"  in  which 
failure  cn  one  mini-test  can  cascade  and  cause 
ether  errors  to  be  reported.  For  example,  e 
procedure  muy  Involve  severel  actions  which  have 
to  be  performed  in  sequence  and  within  a  certain 
time  limit.  If  the  learners  take  too  long  on 
the  first  step,  they'll  also  fail  the  second 
and  subsequent  iteps. 

Another  difficulty  which  can  arise  is  that, 
on  occasion,  the  trainee  will  be  unable  to  perform 
a  required  action  because  another,  more  important 
action  is  required.  This  can  lead  to  double 
bind  situations  in  which  the  system  reports  an 
error  no  matter  wbnt  the  trainee  does. 

Feedback.  Care  must  be  devoted  to  the  design 
of  the  feedback  given  to  the  trainee.  An  automated 
performance  measurement  system  can  detect  every 
minute  violation  of  every  obscure  rule.  If  the 
system  merely  reports  errors,  several  problems 
arise: 

•  The  feedback  can  become  overwhelmingly 
negative. 

•  It  becomes  difficult  for  the-  trainee 
to  appreciate  the-  distinction  between 
serious  errors  and  nit-picking  errors. 

•  If  feedback  is  provided  during  a  performance 
task,  it  can  be  distracting;  yet  if 
provided  after  the  problem,  it  can 
be  difficult  to  correlate  with  behavior, 

•  A  special  problem  which  arises  in  speech 
recognition  based  systems  is  that  perfor¬ 
mance  errors  can  be  accumulated  due 
to  the  fact  that  tht  system  misrecogniscc 
the  trainee's  speech. 

•  The  sheer  number  of  error  reports  encourages 
the  trainee  to  stop  paying  any  attention 
to  them  at  all. 

fdaoUyg. kqfl.CH  I DSJtafcAT.Ul  _  gekq.Q.U<5J! .  We 
have  been  striving  to  develop  an  automated  instruc¬ 
tional  capability  which  can  diagnose  problem 
areas,  prescribe  remedial  work,  recommend  further 
practice  at  a  giver,  level,  and  advance  the  trainee 
to  the  next  learning  task.  A  successful  implemen¬ 
tation  depends  first  of  all  upon  well  validated 
performance  measures.  Secondly,  it  requires 
a  thorough  understanding  of  what  learning  problem 
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p  particular  constellation  of  errors  represents. 
This  is  «t  best  difficult  to  schieve  for  a  oomplex 
performance  task. 

CilBUttC.  Learning  a  per¬ 
formance  task  has  its  purely  oognitlve  aspects, 
but  achievement  of  proficienoy  requires  praotice. 
Often,  supplying  such  practice  can  be  costly 
and  can  even  involve  safety  hazards.  Air  traffic 
controller  training  is  nn  example:  the  novlae 
controller  cannot  be  entrusted  with  responsibility 
for  "live"  aircraft  until  he  or  she  has  attained 
to  a  certain  minimum  level  of  profiolenoy,  and 
yet  practice  is  needed  to  attain  that  proflclancy. 
Simulation  provides  the  obvious  fulfillment  of 
this  training  need. 

Simulations  of  operational  environments 
vary  in  the  degree  of  fidelity  which  they  provide, 
and  the  question  the  training  system  designer 
continual ly  faces  is:  vhut  degree  of  fidelity 
in  simulation  is  required  to  assure  trunsfer 
of  training  to  the  operational  environment? 
To  be  safe,  the  designer  can  speoify  that  a  perfect 
simulation  of  the  environment  is  necessary. 
This  can  not  only  prove  to  be  expensive,  but 
has  been  shown  to  be  unnecessary  in  many  cases. 
Soeie  SOA  Issues  we  consider  in  developing  simulations 
for  training  syntax  ore  described  in  the  paragraphs 
which  follow. 

Environment  Simulation.  Environmental  simu¬ 
lations  need  not  be  extremely  complex  to  provide 
sufficient  fidelity  to  assure  transfer  of  training. 
For  example,  Just  because  the  AIC  must  learn 
to  control  a  variety  of  aircraft  which  differ 
greatly  In  response  characteristics,  it  does 
not  mean  a  training  system  needs  complex  simulations 
of  these  various  aircraft.  The  sweep  rate  on 
the  radar  indicator  is  only  one  sweep  every  twelve 
seconds,  and  a  simple  model  of  aircraft  speed 
and  altitude  and  perhaps  radar  cross  section 
provides  all  the  information  the  AIC  can  normally 
distinguish  on  the  operational  gear.  Thus  a 
relatively  simple  environmental  simulation  car 
support  training  in  a  cost-effective  way.  FUrthet- 
more,  aspects  of  the  environmental  simulation 
can  easily  be  manipulated  to  vary  the  difficulty 
of  the  training  problem  in  a  way  that  could  not 
be  done  at  all  in  the  operational  environment. 
For  example,  in  the  precision  approach  radar- 
controller  training  system,  wind  speed,  direction, 
arid  gusting  characteristics  were  varied  adaptively 
because  this  controller's  job  becomes  more  difficult 
in  windy  conditions.  In  the  AIC  training  system, 
more  bogeys  (bad  guys)  could  oe  added  to  increase 
problem  difficulty  without  the  expense  of  sending 
up  more  F-ks. 

Team  Member  Simulations.  The  simulation 
of  the  other  human3  who  interact  with  the  learner 
is  one  of  the  more  interesting  training  system 
design  problems.  The  traditional  approach  of 
using  human  pseudo  pilots  is  not  necessarily 
the  most  effective  because  these  pseudo  pilots 
may  not  have  ween  trained  to  respond  as  a  pilot 
would.  Often  the  trainees  themselves  take  turns 
serving  as  pseudo  pilots.  This  Is  good  insofar 
as  the  trainee  gets  some  sense  of  what  it  is 
like  to  be  on  the  receiving  end  of  control  infor¬ 
mation,  but  It  can  be  less  helpful  when  one  trainee 
acting  as  a  pseudo  pilot  decides  to  make  life 
either  easy  or  miserable  for  his  buddy.  When 
the  human  pseudo  pilot  is  replaced  by  a  computer 


simulation,  control  oan  be  axcrclaad  ovar  this 
facet  of  tha  environment.  In  the  precision  approach 
radar  oontrollar  training  system  for  example, 
pilot  ability  was  an  adaptive  variable.  When 
the  trainee  was  Just  beginning  to  learn  tha  task, 
a  vary  good  pilot  was  provided.  Aa  tha  oontrollar' a 
proficiency  lnoraaaad,  a  few  poor  pilots  wara 
introduced  to  give  the  trainee  experience  in 
dealing  with  a  variety  of  oontrol  situations. 

Tha  modelling  of  these  other  persons  in 
:he  environment  requires  that  mini  bosk  analyses 
of  their  respective  Jobs  be  conducted  as  well. 
The  oost  of  this  analysis  is  easily  offset  by 
eliminating  the  need  for  pseudo  pilots,  and  a 
significant  training  advantage  is  gained  in  being 
able  to  oontrol  the  responses  of  these  simulated 
individuals. 

/lu.tppk.ted.  .0p.d.erAtw>4iHg 

Speech  understanding  is  «  far  more  difficult 
technical  problem  than  speech  generation.  There 
are  two  main  aspects  to  it:  speeoh  recognition 
(performed  more  or  less  accurately  by  hardware 
devices),  and  speech  understanding  (the  process 
of  mrkjng  an  appropriate  verbal  or  other  response 
hssed  upon  the  speech  input). 

The  advent  of  the  speech  recognition  technology 
has  led  us  into  new  realms  of  adventure  in  training 
automation.  It  made  it  possible  in  theory  to 
automate  the  training  of  the  primarily  verbal 
tasks  of  the  air  traffic  controller.  If  the 
computer  can  understand  the  controller's  advisories, 
It  can  effectively  simulate  pilot  responses  ar.d 
obviate  the  need  for  human  pseudo  pilots  in  training. 
In  our  attempts  to  do  this,  we  have  learned  a 
bit  about  how  to  evaluate  a  speech  understanding 
application  on  the  one  hand  and  how  to  assess 
hardware  speech  recognition  systems  on  the  other. 
The  speech  understanding  concerns  are  discussed 
immediately  below.  Speech  recognition  device 

concerns  were  presented  in  the  NEW  HARDWARE  TO 
USE  section  under  Speech  Hardware. 


Speech  understanding,  as  defined  above, 
refers  to  the  entire  process  of  responding  intel¬ 
ligently  to  speech  Inputs.  This  involves  determining 
what  was  actually  spoken,  and  then  generating 
appropriate  verbal  replies  and  queries,  and  modifying 
the  simulated  envlrorment.  The  problem  of  determining 
what  was  actually  spoken  is  necessitated  by  the 
fact  that  recognition  errors  will  occur.  Manu¬ 
facturer's  claims  of  99+>  speech  recognition 
accuracy  will  probably  not  be  realized,  especially 
In  the  training  environment  where  the  trainee 
is  just  learning  a  verbal  task.  Thus,  it  is 
crucial  for  the  speech  understanding  algorithm 
to  expect  misrecognitions  and  to  check  on  the 
reasonableness  of  any  input  before  ta’-.ing  action 
on  it. 


In  our  experience  in  the  automation  of  this 
..ind  of  training,  where  the  trainee  is  engaged 
in  a  performance  task  and  the  computer  is  providing 
the  simulated  environment,  there  is  a  great  deal 
of  information  in  the  system  which  oan  be  used 
to  check  the  reasonableness  of  an  input  and  either 
automatically  correct  the  input  or  request  that 
the  trainee  "say  again."  This  latter  technique 
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has  to  ba  uatd  with  caution,  for  thara  is  nothin* 
■ore  frustrating  than  hearing  the  computer  repeatedly 
demand  a  rapaat. 


Whan  tha  sy steal  designer  is  faoad  with  the 
need  to  incorporate  a  continuous  speech  recognition 
devioe,  tha  speach  understanding  problem  ba  cones 
vastly  more  ooaplex.  The  training  system  developer 
must  be  aware  of  this  fact.  Tha  decision  to 
use  suoh  a  devioe  oust  be  based  not  only  upon 
user  acoeptanoe  considerations,  but  also  upon 
ability  to  eaploy  talented  software  engineers 
to  design  the  speeeh  understanding  software  and 
the  ability  to  ooamit  significant  computational 
resources  to  the  speech  understanding  prooess. 


t. 
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CONCLUSION 

The  above  discussion  of  pitfalls  and  troll 
bridges  In  SOA  CBI  development  is  not  nearly 
comprehensive,  as  it  refleots  only  our  particular 
experience  with  this  field.  It  doea  show  some 
general  pri  not  pies  to  keep  in  mind:  use  a  systmnatio 
approach,  define  your  need  for  and  uses  of  the 
technologies  at  the  outset;  use  flexible,  oreative 
persons  to  stef  the  projeot;  Insist  on  oroaa 
discipline  oonauntuation  throughout  the  projeot; 
and  leave  plenty  of  reeouroes  for  Iteration  and 
reiteration. 

It's  also  a  good  idea,  when  you  encounter 
problems,  to  try  to  find  someone  who's  been  there 
before  so  you  oan  avoid  reinventing  the  wheel. 
There  will  be  plenty  of  new  mistakes  left  for 
you  to  make. 

State-of-the-art  computer-based  instruction 
systems  represent  a  lot  of  risks  and  a  lot  of 
work.  But,  as  we  have  found  out,  if  you're  diligent 
and  careful,  they're  very  satisfying  to  build 
and  to  use. 
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ABSTRACT 

The  primary  objective  of  the  digital  sensor  simulation  investigations  being  conducted 
at  the  Defense  Mapping  Agency  iDMA)  is  to  establish  an  editing  and  analysis  capability 
for  the  digital  culture  and  terrain  data  bases.  For  purposes  of  quality  control  and 
data  base  applicability  investigations,  DMA  has  developed  the  Sensor  Image  Simulator  (SIS), 
a  very  high  speed  data  base  edit  station  and  static  scene  simulator  that  allows  for 
interactive  query  and  manipulation  of  individual  features  in  the  data  base  displays  and/or 
simulated  sensor  scenes  to  determine  the  corresponding  data  base  elements  responsible 
for  the  simulated  features.  The  SIS  was  installed  at  DMA  in  1981,  and  is  designed  to  play 
a  key  role  in  determining  the  applicability  of  prototype  data  bases  for  use  in  advanced 
training  simulators,  as  well  as  to  ensure  the  quality  of,  and  coherence  between,  the 
various  digital  data  bases  prior  to  new  data  insertion  into  the  master  cartographic 
data  base  files. 


INTRODUCTION 

The  primary  objective  of  the  digital  sensor 
simulation  investigations  being  conducted  at  the 
Defense  Mapping  Agency  (DMA)  is  to  establish  an 
editing  and  analysis  capability  for  the  digital 
culture  and  terrain  data  bases.  These  data  bases 
are  being  produced  by  DMA  to  support  advanced 
aircraft  simulators  by  providing  an  improved 
high,  medium  and  low  level  radar  training  capa¬ 
bility  offered  by  the  digitally  generated  iadar 
Undmass  images.  As  a  result  of  the  technology 
developed  for  the  aircraft  simulator  support, 
sensor  guidance  reference  scenes  are  also  being 
generated. 

In  addition  to  radar  scenes,  visual  and 
multi-sensor  scenes  are  being  digitally  generated. 
For  purposes  of  quality  control  and  data  base 
applicability  investigations,  DMA  has  developed 
the  Sensor  Image  Simulator  (SIS) ,  a  very  high 
speed  data  base  edit  station  and  static  scene 
simulator  that  allows  for  interactive  query  and 
manipulation  of  individual  features  in  the  data 
base  displays  and/or  simulated  sensor  scenes  to 
determine  the  corresponding  data  base  elements 
responsible  for  the  simulated  features  (see 
Figure  1) .  The  SIS  was  installed  at  DMA  in 
1981,  and  is  designed  to  play  a  key  role  in 
determining  the  applicability  of  prototype  data 
bases  for  use  in  advanced  training  simulators, 
as  well  as  to  ensure  the  quality  of,  and  coherence 
between,  the  various  digital  data  bases  prior  to 
new  data  insertion  into  the  master  cartographic 
data  base  files. 

DATA  BASE  CONTENT 

The  current  DMA  standard  production  data  bases 
(Level  I)  contain  large  area  cultural  information 
and  digital  terrain  data  sampled  at  a  3"  interval. 
The  cultural  data  consists  of  point,  linear,  and 
areal  features  described  by  characteristics  such 
as  surface  material  category,  generic  identifi¬ 
cation,  predominant  height,  structure  density, 
and  percentages  of  roof  and  tree  cover.  The 


cultural  data  is  in  lineal  (planimetric  boundary) 
format  and,  although  feature  sizes  may  vary 
depending  upon  local  circumstances,  reflects  a 
resolution  on  the  order  of  500  feet.  Smaller 
features  are  aggregated  into  homogeneous  features 
described  by  predominant  characteristics.  The 
current  high  resolution  (Level  II)  data  bases 
contain  small  area  cultural  information  and 
digital  terrain  elevation  data  sampled  at  a  1" 
interval.  This  translates  to  a  resolution  of 
about  100  feet,  with  smaller  features  aggregated. 
Detailed  information  is  available  in  "Product 
Specifications  for  Digital  Landmass  System  (DLMS) 
Data  Base"  (1) . 

The  terrain  elevation  data  is  produced  by 
contour  digitization  from  charts  or  directly 
from  stereo  pairs  of  photograp' s  using  advanced 
analytical  stereoplotters.  The  cultural  data  is 
produced  from  both  charts  and  photographs  with  a 
much  higher  level  of  manual  effort  required  in 
order  to  perform  the  complex  feature  analysis. 
Because  of  the  labor  intensive  nature  of  the  task, 
the  production  of  Level  II  cultural  data  ranges 
from  10  to  50  times  the  production  cost  of  Level  I 
data,  depending  upon  the  area.  The  current 
Level  I  data  base  program  covers  roughly  24 
million  square  nautical  miles,  with  estimated 
data  base  completion  dates  in  the  1985  to  1995 
time  period.  Level  II  date.  is  programmed  only 
for  small  selected  areas  of  interest. 

The  DLMS  data  bases  have  been  shown  to  be 
adequate  for  support  of  long  and  medium  range 
radar  simulation,  and  for  short  range  radar 
simulation  where  Level  II  data  is  available. 

In  addition,  these  data  bases  have  shown  some 
applicability  for  multi-sensor  simulation  (3,4)  . 

TOE  SENSE  SYSTEM 

SENSE  (2)  is  a  software  package  developed  for 
sensor  simulation  investigations  by  the  Defense 
Mapping  Agency  Aerospace  Center.  A  powerful 
simulation  tool,  it  runs  in  batch  mode  on  one  of 
DMA's  Univac  1100  series  computers, 
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Tho  SENSE  process  begins  by  defining  the  area 
for  which  an  on-line  data  base  is  to  be  constructed. 
Either  center  coordinates  and  areal  extent  or 
limitinq  geographic  boundaries  may  be  entered. 

In  both  cases,  output  spacing  is  specified.  Trans¬ 
formation  to  a  local  coordinate  frame  for  a  number 
of  point  coordinates  may  also  be  specified. 

The  second  step  in  the  process  is  concerned 
with  transforming  the  off-line  data  base  to 
on-line  format.  Two  formats  for  input  terrain 
(i.e.  OEMs  or  DMA  standard)  may  be  accepted.  The 
latitude  and  longitude  boundaries  of  the  area  are 
also  input.  For  the  input  cultural  feature  filesi 
a  variety  of  options  may  be  exercised  to  define 
which  manuscripts  or  features  are  to  be  utilised. 
Synthetic  feature  breakup  of  the  data  base  in 
accordance  with  feature  characteristics  may  also 
be  specified.  Output  displays  for  terrain, 
culture  or  merged  data  may  be  generated  for  the 
line  printer  or  an  Optronics  film  recorder.  The 
principal  output  of  this  step,  however,  is  the 
on-line  data  base  in  blocked  matrix  format. 

Once  generated,  the  on-line  data  base  may  be 
processed  by  four  separate  program  structures) 

(a)  the  data  base  may  be  reblocked  to  provide  an 
on-line  data  base  of  different  spacing,  (b)  it  may 
be  input  to  a  plot  module  to  provide  a  variety  of 
plot  types,  tc)  it  may  be  unblocked  to  provide  a 
single  large  data  base,  and  (d)  it  may  be  used  as 
the  input  for  sensor  simulation  software. 

For  sensor  simulation,  considerable  variation 
in  the  SENSE  output  is  possible  by  appropriate 
selection  of  input  variables.  These  include: 

1.  Input  data  base  characteristics  and  portion 
utilised. 

2.  Sensor  coverage  and  characteristics  (type, 
receiver  characteristics,  antenna  pattern,  etc). 

3.  Constants  defining  sensor  position  and  the 
projection  coordinate  transform. 

4.  Weather  options  and  atmospheric  yariables. 

5.  Output  plot  options. 

A  number  of  deficiencies  result  from  the 
computer  environment  in  which  SENSE  operates: 

1.  DMA's  1100  Series  computers  cannot  be 
totally  dedicated  to  quality  control/validation. 

2.  The  system  is  not  interactive;  editing  is 
therefore  not  feasible  nor  are  the  results  of 
processing  stages  immediately  available. 

3.  Both  the  off-line  to  on-line  transforma¬ 
tions  ar.d  sensor  simulation  tasks  are  slow 
(typically  10-30  minutes) . 

4.  Program  structures  are  not  optimized. 

THE  SIS  CONCEPT 

The  natural  evolution  of  sensor  simulation  at 
DMA  led  to  the  design  and  fabrication  of  the 
Sensor  Image  Simulator  (SIS) ,  a  dedicated  mini- 
computer-based  image  processing  system  capable 
of  performing  simulations  in  an  interactive  mode . 


The  SIS  brings  together,  in  a  self-contained 
integrated  hardware/software  facility,  a 
significant  capability  to  evaluate  the  DLMS  data 
base.  All  operation*  are  conducted  under  inter¬ 
active  control.  Both  the  software  structure 
and  operations  sequence  reflect  a  top-down 
Implementation  philosophy  wherein  principal 
control  function*  are  resident  at  the  top  of 
the  hierarchy  and  functions  concerned  with 
procasaing  individual  data  elements  (I/O, 
computation,  etc.)  are  at  the  lowest.  The  system 
is  implemented  in  such  a  fashion  that  future 
changes  in  processing  can  be  accomplished  at  the 
higheut  level  of  system  software  support. 

Basic  input  data  to  the  system  consists  of 
the  off-line  DLMS  data  base  tapes  and  operator 
commands  apecifying  which  tape  data  available 
is  to  be  transferred  to  secondary  storage  (the 
system  disk) .  Editing  commands  revising  the 
off-line  data  (on  secondary  storage)  may  also  be 
utilised,  and  a  designated  portion  of  the  changed 
data  may  be  transferred  back  to  tape,  with 
appropriate  optional  diagnostics  documenting 
transfers  in  either  direction. 

Once  resident  on  the  system  disk,  any  portion 
of  the  off-line  data  may  be  transformed  into  a 
viewable  on-line  format,  in  order  to  permit 
viewing  of  the  data  base,  a  sensor  module  must  be 
specified  to  transform  feature  data  to  sensor- 
related  quantities  (reflectivity,  albedo,  etc). 

The  local  coordinate  transform  may  also  be 
specified.  The  on-line  version  is  constructed 
in  such  a  fashion  that  the  operator  may  easily 
Interrogate  and  change  the  data  base  and  relate 
these  revisions  to  the  off-line  data  base. 

The  final  step  of  processing  under  normal 
interactive  control  is  concerned  with  3enF.,r 
simulation,  two  types  of  scenes  are  generated 
during  this  stage.  The  first  of  these  (selectable 
by  the  operator)  is  the  perspective  view.  This 
option  permits  selection  of  position  and  line  of 
sight  for  mapping  the  (three  dimensional)  data 
base  into  the  observer's  image  display  coordinate 
frame.  The  second  type  of  scene  generated  is 
the  sensor  display.  Data  used  is  common  to  that 
generated  for  the  perspective  view.  Modifications 
which  may  be  introduced  into  the  sensor  display 
include  sensor  parameters  (e.g.  beam  error)  and 
sensor  display  variations  (e.g.  gain).  An 
important  characteristic  of  the  display  transform 
is  that  the  operator  can  easily  establish  the 
relationship  between  the  sensor  display  and  the 
on-line  data  base. 

In  addition  to  the  processing  stages 
(including  editing) ,  normally  under  operator 
control,  software  development  and  maintenance  is 
also  an  interactive  function  using  the  system’s 
text  editor  and  FORTRAN  compiler. 

SIS  OPERATIONS 

The  Sensor  Image  Simulator  performs  five 
major  functions: 

1.  Digital  Data  Base  pile  input  and  Output. 
The  capability  of  loading  operator  designated 
sections  of  the  DLMS  off-line  data  base  (both 
terrain  and/or  culture  files)  onto  the  system 
secondary  storage  device  is  provided.  The 
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capability  also  exists  to  off-load  such  data 
(including  modifications)  onto  tape  in  DLMS 
standard  format.  A  line  printer  listing  docu¬ 
menting  the  loading  process  may  (at  the  operators 
discretion)  also  be  provided. 

2.  Off-Line  to  On-Line  Transformation. 

The  SIS  system  performs  the  processes  (coordinate 
transformation,  area  fill,  etc.)  necessary  to 
transform  the  off-line  DLMS  data  base,  stored  as 
described  above,  into  an  on-line  format  capable 
of  being  viewed,  and  modified  by  the  operator, 
as  well  as  used  in  subsequent  stages  of  processing. 
The  transformation  is  accomplished  in  approximately 
two  minutes  for  a  l“x  1®  Level  I  area. 

3.  Sensor  Simulation.  The  SIS  facility 
generates  simulated  sensor  displays  using  the 
on-line  data  base.  The  system  is  structured  such 
that  data  from  intermediate  stages  of  processing 
is  available  for  viewing  and/or  the  introduction 
of  operator  controlled  variations. 

4.  Interactive  Data  Base  Editing.  As  a 
quality  control  tool  for  DLMS,  the  SIS  provides 
the  operator  with  the  ability  to  both  define  the 
characteristics  of  and  modify  the  three  principal 
data  structures  present  (off-line  DLMS,  on-line 
DLMS,  and  the  sensor  display) . 

5.  Software  Development  and  Maintenance. 

In  order  to  provide  the  future  evolution  of 
sensor  simulation,  the  SIS  is  structured  to  permit 
changes  in  function  by  modification  of  the 
appropriate  software  module/sub-modules  affected. 
Whole  modules  may  also  be  inserted  or  removed. 

This  allows  for  variation  in  future  data  base 
formats.  The  commercially  supported  real  time 
multi-tasking  operating  system  supports  software 
development  via  system  utilities  (text  editor, 
assembler  and  FORTRAN  compiler) . 

SIS  HARDWARE 

The  SIS  hardware  may  be  discussed  in  four 
general  areas  (6) . 

1.  Host  Computer.  The  host  computer  is  a 
Data  General  "Eclipse"  S/250  with  integral  array 
and  floating  point  processors. 

2.  Soft-copy  Image  Display  Subsystem. 

Data  base  and  sensor  images  are  displayed  on 
either  the  Aydin  8026  Color  Monitor  or  the 
Aydin  8037  Monochrome  Monitor.  Both  units 
display  1024  x  1024  images  and  are  controlled 
by  the  Aydin  5116  Display  Editor  Keyboard  and 
Joystick  and  the  Aydin  5216  Display  Computer. 
Graphic  data  may  be  displayed  on  the  Tektronix 
4000-1  Graphic  Display  Terminal . 

3.  Hard-copy  Image  Display  Subsystem. 

Color  hardcopy  images  are  obtained  from  the 
Matrix  Instruments  4007  Color  Graphic  Camera 
yielding  both  8  in.  x  10  in.  instant  copy  and 
35mm  film  output.  For  quick  inexpensive 
monochrome  copy,  the  Tektronix  4634  Video  Hard 
Copy  Unit  is  used. 

4.  Peripheral  Devices.  The  SIS  config¬ 
uration  is  completed  by  the  following 
peripheral  devices: 


a.  Analogic  AP400  Array  Processor. 

b.  Data  General  6026  1600/800  FPI  Tape 

Unit. 

c.  AVIV  TFS  706-125  6250/1600/800  FPI 
Tape  Unit. 

d.  Data  General  6061  190  MB  Disc 
Storage  Unit. 

e.  Data  General  6070  20  MB  Disc 
Storage  Unit. 

f.  Data  General  6040  Terminal  Printer. 

g.  Teletype  40  Line  Printer. 

SIS  SOFTWARE  ARCHITECTURE 

The  development  of  software  for  SIS  was 
determined  by  the  following  requirements : 

1 .  The  software  structure  must  be  user 
programmable  to  support  changing  data  base  and 
sensor  support  requirements. 

2.  The  throughput  capabilities  of  the  hard¬ 
ware  elements  of  the  system  must  be  accessible 
at  the  highest  levels  of  software  development. 

3.  User  interaction  with  the  system  must  be 
accomplished  in  such  a  fashion  that  personnel  not 
intimately  familiar  with  digital  processing  are 
capable  of  utilizing  its  functions. 

Orderly  development  of  SIS  software  was 
accomplished  via  a  top-down  implementation 
philosophy  wherein  the  software  structure  is 
strongly  reflected  in  the  operations  sequence. 

All  application  functions  are  accomplished  via 
interactive  control.  The  hierarchial  system 
structure  guides  the  operator  in  defining  a 
processing  requirement  with  an  increasing  level 
of  detail  until  all  necessary  parametric  entries 
are  available .  The  defined  processing  then  takes 
place,  after  which  the  operator  is  given  the 
choice  of  returning  to  a  higher  level  in  the 
system  or  repeating  the  processing  sequence  with 
a  redefined  set  of  parameters. 

Accommodating  a  variety  of  users  within  the 
system  has  led  to  the  development  of  entry  and 
menu  display  software  which  combines  fail-safe 
operation  and  comprehensive  explanation  capabil¬ 
ities  with  features  designed  to  reduce  redundancy 
for  highly  knowledgeable  personnel .  Although  the 
principal  application  language  is  FORTRAN,  the 
decoding  of  operator  entries  is  handled  via 
application  routines  rather  than  standard  FORTRAN 
formatted  I/O.  Thus,  the  detection  of  errors  may 
be  handled  by  the  application  program,  rather 
than  at  the  systems  level  thereby  avoiding 
execution  abort  due  to  input  error.  In  addition, 
the  operator  is  given  the  opp-.rtunity  to  revise 
entries  before  proceeding  (via  a  simple  carriage 
return  on  the  system  console) .  At  the  level  where 
a  large  number  of  entries  (each  with  a  variety  of 
possible  settings)  is  required,  submenus  for  each 
parameter  are  not  initially  displayed  unless 
requested  by  the  operator  (by  simply  entering  the 
parameter  number) .  In  addition,  an  entry  of 
several  parameter  values  may  be  made  simultaneously 
with  repeated  menu  displays.  At  the  same  time,  a 
detailed  explanation  of  possible  entry  values  for 
various  parameters  is  available  directly  at  the 
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system  console  via  a  simple  procedure  for 
each  level  of  the  process. 

CONCLUSIONS 

The  Sensor  Image  Simulator  is  designed  to  play 
a  key  role  in  both  the  requirements  analysis  and 
quality  control  of  the  DLMS  data  bases  being 
produced  by  DMA.  Current  investigations  are 
allowing  for  the  development  of  production 
quality  control  scenarios  to  maximize  the 
capabilities  offered  by  the  SIS,  The  usefulness 
of  the  SIS  is  being  expanded  through  the 
development  of  new  sensor  modules  designed  to 
determine  data  base  requirements  for  a  variety 
of  sensors. 
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ABSTRACT 


With  the  development  of  increasingly  complex  weapon  systems  comes  the  need  to  train 
personnel  to  operate  these  systems,  which  now  require  an  Increasing  array  of  aircrew  skills. 
This  paper  focuses  attention  on  the  need  for  development  of  Part  Task  Trainers  (PTT)  which 
enable  aircrews  to  acquire  these  skills  In  a  timely  and  cost-effective  manner.  It  presents 
a  multidlsclpllne  view  of  PTTs,  as  perceived  by  an  Engineering  Psychologist,  a  Fighter  Pilot, 
and  a  Design  Engineer.  It  briefly  reviews  the  record  of  PTT  development,  focusing  on  the 
Importance  of  front-end  analysis  of  training  requirements  as  the  basis  for  considering  and 
selecting  training  approaches  that  may  effectively  be  met  by  PTTs.  Target  areas  for  PTT 
applications  are  discussed  and  proposals  are  offered  for  the  development  of  some  unique 
PTT  concepts  Including  the  potential  for  Increased  use  of  generic  and  specialized  PTTs. 
Potential  solutions,  cost  savings  benefits,  and  Improved  training  expectations  are  discussed. 


INTRODUCTION 

Aerospace  weapons  systems  (and  military  equip¬ 
ment  In  general)  are  becoming  Increasingly  complex 
as  growing  technology  provides  Increased  total 
system  capability.  Modern  military  aircraft 
employ  a  sophisticated  complex  Integration  of 
digital  avionics,  sensor  systems,  fire  control  and 
weapons  delivery  systems.  The  degree  of  complex¬ 
ity  Is  Illustrated  by  aircraft  such  as  the  F-15, 
F-16,  and  F-18;  sensor  systems  such  as  PAVE  TACK 
and  LANTIRN;  and  Precision  Guided  Munitions  (PGM) 
such  as  the  GBU-15  Guided  Bomb  and  AGM-65  Maverick 
Air-to-Ground  Missile. 

Operation  of  these  complex  systems  requires 
an  increasing  array  of  cognitive,  manipulative 
and  psycho-motor  skills.  Methods  for  training  the 
associated  tasks  have  received  considerable  atten¬ 
tion,  but  only  recently  has  much  notice  been  given 
to  organizing  the  tasks  so  that  specific  behaviors 
are  treated  In  specific  devices.  For  a  long  time, 
the  main  thrust  of  trainer  design  has  been  toward 
designing  for  maximum  training  potential .. .and  a 
much  worn  out  phrase. . .just  like  the  aircraft. 

The  response  to  simulator  (training)  requirements 
for  the  newer  sophisticated  aircraft  has  also  been 
generally  to  produce  simulators  which  will  fly 
like  the  aircraft.  Lacking  sufficient  quantitative 
data  on  training  effectiveness,  simulator  users 
and  developers  have  played  it  safe  and  opted  for 
maximum  fidelity  In  simulator  engineering. 

Simulator  technology,  particularly  In  the  visual 
area,  has  made  rapid  advances  and  some  current 
simulator  programs  are  pushing  the  state-of-the-art, 
sometimes  beyond  technology  limits.  Simulators, 


therefore,  have  become  much  more  complex  and 
sophisticated. .  .and  Increasingly  expensive.  The 
new  B-52/KC-1 35  Weapon  Systems  Trainers  (WSTs)  are 
examples  of  highly  complex  training  devices  which 
are  representative  of  current  state  of  the  art  for 
full-mission  training.  The  approach  to  simulating 
various  subsystems  for  electro-opt1cal(E0)/infrared 
Imaging,  fire  control  and  weapons  delivery  has  also 
been  driven  by  a  similar  desire  to  replicate  the 
"real-world."  Without  definitive  analysis  to  prove 
otherwise,  a  fully  Integrated  system  has  generally 
been  the  requirement.  This  approach  Is  usually 
based  on  the  general  user  feeling  that  only  a 
"fully  operational"  cockpit  will  provide  effective 
training. 

Recent  trends  in  training  systems  design  have 
therefore  emphasized  the  large  WST  systems  with 
intensely  accurate  visual  and  sensor  simulation 
subsystems  that  attempt  to  emulate  the  "real-world" 
environment.  This  can  result  In  a  costly  develop¬ 
ment  and  production  program  when  (for  example) 
Forward  Looking  Infrared  (FLIR)  or  Digital  Radar 
Landmass  Simulation  (DRLMS)  Is  required  to  be  Inte¬ 
grated  into  an  existing  simulator  to  produce  full- 
mission  simulation  over  an  extensive  gaming  area. 

We  believe  that  not  enough  consideration  has  been 
given  to  identifying  related  sets  of  training 
requirements  suitable  for  smaller,  less  complex 
devices  to  be  used  as  PTTs.  This  paper  focuses 
attention  on  the  possibilities  for  part-task 
training  to  cost-effectively  train  many  of  the 
unique  and  complex  tasks  associated  with  the  opera¬ 
tion  of  modern  aircraft  and  their  weapons  systems. 
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Basically,  the  USAF  defines  the  use  of 
trainers  Into  several  categories.  Weapon  Systems 
Trainers  (WSTs)  are  the  most  complex  (and 
expensive)  simulators,  but  they  permit  each  crew 
member  to  practice  a  full -mission  task  and  become 
proficient  In  a  very  realistic  environment. 
Operational  Flight  Trainers  (OFTs)  provide 
training  In  basic  flight  maneuvers  and  aircraft 
systems  operation  as  well  as  normal  and  emergency 
procedures.  Cockpit  Procedures  Trainers  (CPTs) 
permit  pilots  to  develop  proficiency  In  routine 
procedures  and  to  react  to  emergency  procedures 
like  engine  failures,  fire  or  electrical  malfunc¬ 
tions.  Cockpit  Familiarization  Trainers  (CFTs) 
provide  basic  practice  on  location  of  switches  and 
Indicators,  and  are  normally  used  to  Initiate 
pilot  training  for  a  particular  aircraft.  Part 
Task  Trainers  (PTTs)  permit  crew  members  to  master 
and  maintain  currency  In  specific  tasks  like 
aerial  refueling,  gun  sight  dynamics,  sensor  system 
Interpretation  or  system  tuning.  Such  mastery  can 
often  be  achieved  In  less  complex  trainers  more 
effectively  than  using  a  single  WST  or  OFT.  This 


becomes  particularly  Important  when  acquisition 
and  maintenance  costs  limit  the  quantity  of 
trainers  and  result  In  training  "bottlenecks." 

An  operational  definition  that  we  have  come 
to  use  for  PTTs  Is:  a  device,  limited  In  scope, 
that  Is  designed  to  treat  a  specific  set  of 
behavioral  events  that  will. ..at  some  time. ..be 
Integrated  with  other  behaviors  to  perform  a 
broader  task.  Part  Task  Trainers  are  not  new; 

In  fact,  they  represent  possibly  the  earliest  form 
of  "active  participation"  training  aid  to  be 
employed  (WWI  Aerial  Gunnery).  Groups  of  PTTs 
have  been  known  as  trainer  "families,"  'building 
blocks,"  and  more  recently  "modular"  approaches 
to  training.  Some  recent  PTTs  have  also  become 
large  systems,  nearly  as  complex  as  WSTs .. .because 
of  the  extensive  list  of  cues  apparently  required 
by  the  training  task.  An  example  of  a  complex  PTT 
Is  the  B-52  Aerial  Refueling  PTT  (by  Redlfon), 
which  has  a  full  cockpit  with  a  Duovlew  visual 
display  system  mounted  on  a  hydraulic  motion  base 
(Figure  1 ) . 


Figure  1.  B-52  Aerial  Refueling  Part-Task  Trainer 
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The  KC-1 35  Boom  Operator  PTT  (by  ASD  and  SRL), 
with  a  true  perspective  visual  display  and  range 
bed  camera/model  Image  generator,  represents  an 
Intermediate  level  of  Pf T  complexity  (Figure  2)(4). 
The  Functional  Integrated  Systems  Trainer 
(FIST)  (by  SRL)  for  the  AC-130H  Gunshlp  Is 
another  example  of  a  medium  complexity  PTT 
(Figure  3)(2).  This  PTT  Is  somewhat  unique 
because  It  provides  for  simultaneous  training  of 
tasks  associated  with  several  crew  stations, 
and  Is  the  equivalent  of  multiple  Interconnected 
part  task  trainers.  The  recently  delivered  F-106 
Aerial  Gunnery  PTT  (AGPTT)  (by  Honeywell) 
represents  a  part- task  training  system  of  modest 
complexity  (Figure  4).  Prior  to  AGPTT  development 


successful  construction  and  use  of  the  MA-1  Fire 
Control  System  PTT  for  F-106  aircraft  demonstrated 
a  high  training  value  In  proportion  to  overall 
size  and  facility  requirements  (Figure  5 ) (1 6 ) . 

The  prototype  ARN-101  Navigation  System  PTT  for 
the  F-4  aircraft  represents  a  smaller,  but  fully 
Interactive  training  device  (Figure  6).  This  PTT 
Is  an  attachment  for  existing  ARN-101  mission  data 
preparation  consoles,  utilizing  spare  computer 
capacity  for  training  the  aircraft  system 
operation.  The  smallest  part  task  training  devices 
are  represented  by  the  desktop  EW  trainers  produced 
by  several  different  companies.  They  allow  Inter¬ 
active  student  Instruction  not  possible  with  video 
tape  or  classroom  lectures. 
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Note:  Trainer  prototype  Is  attached  as  an  optional 
peripheral  to  the  ARN-101  Mission  Data  Transfer 
System  ( MDTS )  housed  In  the  portable  travel  case. 
The  trainer  can  be  disconnected  to  allow  Immediate 
MDTS  deployment  with  associated  aircraft. 


Figure  6A,8.  ARN-101  Navigation  System  Trainer  Prototype 


In  a  time  when  worldwide  commitments  for  short 
notice  deployment  are  typical  of  operational 
requirements,  the  status  of  crew  dally  mission 
readiness  could  be  determined  by  the  regular  use 
of  a  capable,  effective  training  system.  The 
capability  of  the  training  system  Is  frequently 
driven  by  cost  considerations  and  the  time 
required  to  manufacture  and  deliver  a  system.  The 
future  looks  like  a  move  is  appropriate  toward 
more  low  cost  devices  available  to  more  students 
simultaneously.  This  approach  would  enable  better 
use  of  fewer  high  cost,  full-mission  trainers. 

By  using  a  family  of  modular  training  devices, 
the  training  and  operating  commands  could  have  a 
better  potential  for  keeping  the  air  crews  mission 
ready,  while  reducing  the  training  backlog  created 
by  the  part  task  use  of  scarce  WSTs  (where  avail¬ 
able)  and  eliminating  the  reluctant  necessity  to 
do  "training"  In  powered-up  aircraft  parked  on  the 
ground. 

THE  FRONT  END  PROCESS 

From  the  early  start  of  the  Systems  Approach 
to  Training  (SAT)  (19)  In  the  Air  Force  through 
growth  of  the  Instructional  Systems  Development 
Process  (ISD)  (24),  emphasis  has  been  given  to 
development  of  the  academic  process.  Analysis 
of  training  requirements  concerned  teaching  steps, 
course  content,  supporting  equipment  and  media 
mix.  From  some  of  the  comments  by  Montemerlo  (13). 
the  Navy  took  similar  approaches.  Acquisition 
managers  picked  up  on  the  process  and  wrote 
training  requirements  Into  need  documents  as 
though  the  ISD  process  was  designed  to  describe 
training  In  terms  of  specific  equipment  and  techni¬ 


cal  approaches  to  trainer  design.  The  ISD  task 
analysis  was  not  designed  to  define  performance 
specifications  for  training  devices.  As  a  result, 
much  Interaction  has  occurred  In  each  of  the 
services  about  how  to  accurately  and  completely 
provide  training  requirements  to  trainer  acquisi¬ 
tion  managers  In  terms  useable  by  design  engineers 
and  program  managers.  Additionally,  we  In  the 
USAF  simulator  acquisition  business  have  sometimes 
found  it  difficult  to  get  the  simulator  user  to 
choose  a  set  of  training  tasks  (or  cues)  realiz¬ 
able  with  near-term  technology. . .and  within 
available  fiscal  constraints.  Although  the 
acquisition  management  process  has  been  amply 
"procedureized,"  the  needed  work  to  tie  training 
analysis  and  acquisition  management  processes  for 
training  devices  together  Is  still  In  a  growing 
stage. 

Growing  Interests  In  clarifying  Initial 
descriptions  of  training  requirements  and  updating 
on-going  training  has  come  from  In-depth  studies 
of  the  analysis  process.  The  current  term  for 
this  evolving  process  Is  being  called  front-end 
analysis  (FEA)  (8,  14,  15,  18,  23).  In  fact,  what 
Is  happening  Is  a  closure  of  awareness  between 
training  managers,  design  engineers  and  acquisi¬ 
tion  managers  of  the  varying  approaches  each  takes 
toward  doing  their  part  of  delivering  a  useable 
training  device.  By  somewhat  getting  Into  each 
other's  business,  we  are,  In  fact,  beginning  to 
understand  the  larger  process. 

Since  training  analyses  were  not  designed  to 
specifically  describe  training  devices,  a  holistic 
view  has  existed  that  designing  a  trainer  to  meet 
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most  training  event  requirements  was  the  best 
approach.  This  generally  led  to  the  evolution  of 
designs  for  OFTs  and  WSTs. 

For  several  existing  weapons  systems,  PTTs 
were  designed  to  deal  with  specialized  training 
needs  that  either  could  not  be  satisfied  In  the 
MST  or  that  saturated  the  WST  schedule  while 
doing  skill  level  training.  As  overall  skill 
levels  Increased,  training  managers  found  the 
PTTs  to  be  useful  assets  and  relatively  easy  to 
maintain.  Successful  use  of  these  PTTs  (such  as 
the  F-106  Aerial  Gunnery  PTT)  has  spurred  many 
discussions  andproposals  for  use  of  PTTs,  and  has 
caused  the  Aeronautical  System  Division's  Deputy 
for  Simulators  (ASD/YW)  to  place  more  emphasis  on 
the  use  of  PTTs  as  alternative  solutions  to 
training  requirements.  Additionally,  the  current 
economic  situation  Is  driving  consideration  of 
low  cost  training  devices  as  Interim  solutions  to 
training  requirements. 

The  USAF  approach  to  developing  technical 
solutions  to  training  requirements  has  been 
couched  In  a  multi -discipline  team  approach  to 
provide  functional  specialists  to  analyze  the 
requirements.  The  team  performs  an  evaluation  at 
an  early  stage  In  evolution  of  the  weapon  system 
or  changing  mission.  This  team  then  utilizes 
their  collective  experience  and  the  wealth  of 
available  Information  on  simulator  design  to  look 
for  Innovative  solutions  to  training  requirements. 
In  addition,  they  are  guided  by  documented  policy 
to  ensure  that  Air  Force  goals  In  training, 
supportabllity,  costs,  etc.  are  met.  A  program 
Is  now  underway  to  develop  a  systematized  concept 
for  analyzing  training  requirements  and  comparing 
the  analysis  to  various  technical  solutions. 

Using  models  that  may  be  automated,  an  Iterative 
cycle  of  analyze-compare-analyze-compare  will 
occur  until  functional  managers  are  satisfied  that 
the  best  alternatives  have  been  described. 

It  Is  In  participating  In  the  FEA  process  that 
the  authors  have  perceived  the  forthcoming  Impact 
of  digital  avionics  and  "smart"  weapons  systems  on 
training  requirements,  have  seen  the  opportunity 
for  PTTs  to  meet  these  requirements  and  have  seen 
the  need  for  the  tri-servic.es  and  Industry  to  look 
for  some  common  ground  In  the  development  of 
training  devices  for  these  systems.  As  a  start 
point,  we  now  need  to  consider  some  of  the  tasks 
and  system  applications  common  to  modern  weapons 
systems  which  are  particularly  applicable  to  PTT 
solutions. 

SOME  TARGET  AREAS  FOR  PTTs 

Many  of  the  newer  airborne  weapons  systems 
possess  a  high  degree  of  complexity  but  have 
similar  components.  Features  such  as  digital 
entry  keyboards,  displays,  computers,  laser  range¬ 
finder/designators,  radars,  and  Inertial  navigation 
Interfaces,  provide  some  possibilities  for  cost 
effective  combinations  or  re-use  of  engineering 
effort  to  model  and  simulate  one  particular  system. 
Most  new  systems  are  also  controlled  by  a  digital 
computer,  which  can  facilitate  the  resolution  of 
Interface  problems.  These  weapons  systems  are 
also  Intended  to  be  "modular"  or  "add-on"  In 
nature,  a  feature  that  should  also  land  Itself  to 
the  concept  of  a  PTT  for  the  weapon.  To  Illustrate 
possible  use  of  PTT's  for  these  systems,  some 
selected  tasks  and  system  applications  are 


described  In  the  following  paragraphs. 

Task  Dlfflculty/Tralnlng  Time: 

In  modern  aircraft,  despite  the  technological 
sophistication,  or  perhaps  because  of  It,  there 
are  an  Increasing  number  of  tasks  which  take  a 
disproportionate  amount  of  time  to  train.  In  other 
words,  a  student  will  spend  extra  missions  in  a 
simulator  to  reach  the  required  performance  stand¬ 
ard  In  a  few  tasks,  each  of  which  may  occupy  only 
a  short  time  In  the  real-world  mission.  Addition¬ 
ally,  the  demands  of  the  more  difficult  tasks  may 
be  such  that  the  student's  necessary  attention  to 
other  tasks  Is  degraded,  thereby  reducing  the  total 
value  of  the  training  session.  Some  of  these  tasks, 
such  as  air-to-air  target  tracking  and  air-to- 
ground  weapons  deliveries,  traditionally  require 
extensive  training  In  the  aircraft.  With  modern 
simulator  technology  they  can  be  trained  In  a 
simulation  device,  and  because  of  their  criticality 
and  required  training  time,  might  well  be  trained 
In  a  PTT.  Other  unique  tasks  which  require 
considerable  training  time,  mainly  In  Initial 
training.  Include  Heads  Up  Display  (HUD)  Interpre¬ 
tation,  Hands  on  Throttle  and  Stick  (HOTAS) 
operation,  radar  and  Infrared  (IR)  Image  interpre¬ 
tation,  PGM  operation  and  control,  Inertial 
Navigation  Set  (INS)  operations,  and  Electronic 
Warfare  (EW)  operations.  These  are  the  more 
obvious  tasks;  for  each  aircraft  there  may  be 
unique  tasks  which  require  extensive  training  time. 
Provision  of  PTTs  for  these  types  of  tasks  should 
reduce  total  WST  training  time  and  therefore  reduce 
required  WST  numbers.  However,  analysis  of  student 
throughput  rates,  task  training  times,  training 
device  costs,  Initial  versus  continuation  training, 
etc.  will  be  needed  to  provide  the  correct  mix  of 
training  devices. 

Visual  Discrimination  Tasks: 

There  are  many  operational  tasks  which  require 
a  high  degree  of  visual  discrimination.  While  some 
aircrew  members  may  have  a  greater  Inherent  ability 
for  visual  discrimination,  the  more  difficult 
processes  have  to  be  learned  through  practice  and 
knowledge  of  the  mechanisms  by  which  the  visual 
i.iwge  Is  received.  These  visual  discrimination 
tasks  Involve  out-the-wlndow  scenes  as  well  as 
Imagery  presented  on  various  cockpit  displays  such 
as  radar  and  IR.  Although  out-the-wlndow  discrimi¬ 
nation  tasks  such  as  low-level  flying  could  merit 
some  discussion,  we  will  consider  only  the  on-board 
Imagery  such  as  radars,  EO,  and  IR.  In  general, 
the  development  of  high  fidelity  simulation  of  this 
type  of  Imagery  Is  extremely  expensive.  However, 
depending  on  a  complete  analysis  of  the  primary 
task  requirements  and  any  associated  tasks,  It  may 
be  possible  co  train  the  basic  task  on  a  PTT, 
using  such  media  as  slides,  videotapes,  movies  or 
a  combination  of  media.  A  complete  training 
program  including  lecture,  written  and  PTT  training 
will  Insure  that  the  aircrew  member,  upon  reaching 
the  simulator  or  aircraft,  does  not  spend  a 
disproportionate  amount  of  time  on  visual  discrimi¬ 
nation  tasks. 

Further  complications  arise  when  considering 
the  provision  of  simulated  radar  and  IR  Imagery  In 
a  WST.  The  question  of  fidelity  becomes  vital, 
because  for  some  visual  discrimination  tasks, 
negative  training  could  result  from  poor  simulation. 
State-of-the-art  radar  simulation  is  generally 
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regarded  as  providing  positive  training  for  the 
radar  discrimination  task;  FUR  Interpretation 
Is  a  more  difficult  technical  challenge  to  train 
by  simulation.  "Real-world"  IR  Imagery  has  a  high 
dynamic  range,  and  the  variable  scene  conditions, 
weather  effects,  time  of  day.  seasonal  effects, 
and  other  peculiarities  are  very  difficult  to 
simulate  fully.  At  present,  no  system  can 
provide  full  training  for  IR  Imagery  Interpreta¬ 
tion  while  simultaneously  allowing  random  low- 
level  flight  over  a  large  gaming  area. 

It  may  be  more  cost  effective,  or  simply 
provide  better  training,  to  use  a  PTT  device  for 
IR  Imagery  which  uses  videotape  or  film  to  provide 
real-world  training  In  IR  Interprc  atlon  for  a 
limited  gaming  area.  Further  analysis  will  Indi¬ 
cate  the  degree  of  Interaction  of  other  tasks 
with  the  discrimination  task.  Frequently  there  Is 
a  requirement  to  lock-on  to  an  Image  so  that 
lock-on  parameters  can  be  fed  to  the  aircraft's 
navigation  system.  The  trainee  may  learn  to  lock- 
on  to  a  representative  Image  In  the  WST  while  the 
target  discrimination  task  Is  trained  separately 
un  a  PTT.  Alternatively  for  full -mission  training 
In  a  WST,  It  may  be  possible  to  reduce  the  cost  of 
radar  and  IR  simulation  by  confining  the  high 
fidelity  simulation  to  limited  gaming  areas  or 
corridors,  with  a  lower  fidelity  generic  simula¬ 
tion  elsewhere,  so  that  the  trainee  having 
learned  target  discrimination  on  a  PTT,  need  spend 
little  time  actively  performing  this  task  In  the 
WST.  For  simulator  mission  training  and  evalua¬ 
tion,  the  aircrew  need  oniy  perform  the  task  In 
defined  critical  areas  such  as  navigation  way 
points  and  target  discrimination,  it  may  also  be 
cost  effective  to  use  the  aircraft  system  for 
final  training  and  validation.  Finally,  as  will 
be  discussed  later,  there  are  a  number  of  PGMs 
which  utilize  visual  displays,  and  whether  Inte¬ 
grated  In  a  WST  or  as  stand  alone  PTTs,  the 
requirement  to  train  the  visual  discrimination 
task  '.'111  need  a  critical  analysis.  Many  of  the 
above  considerations  could  well  apply. 

Mission  Critical  Tasks: 

Part  task  training  Is  particularly  applicable 
to  mission  critical  tasks.  These  are  tasks  where 
successful  task  execution  at  the  first  attempt 
Is  essential  to  mission  accomplishment,  for 
reasons  of  surprise,  hostile  environment,  or  other. 
The  following  examples  Illustrate  some  mission 
critical  tasks  suitable  for  PTT  training:  PGM 
terminal  delivery/guidance;  electronic  warfare 
terminal  threat  warning  recognition;  air-to-air 
gunnery/miss  He  engagements;  and  FLIR/radar  target 
Identification. 

With  modern  PGMs,  strike  aircraft  will  carry 
fewer  of  these  complex  weapons  (due  to  weapon  cost 
and  compatible  stores  position  availability),  so 
the  accurate  delivery  of  a  single  weapon  Is 
Inherently  mission  critical.  Targets  that  would 
previously  have  required  multiple  passes  with 
conventional  bombs  to  assure  destruction,  must  be 
“taken  out"  by  a  single  precision-guided  weapon. 
Another  mission  critical  task  Involves  the  rapid 
recognition  of  a  surface-to-air  missile  (SAM) 
radar  mode  change  from  search-to-track  (or 
especially  from  track-to-launch)  that  may  be 
required  to  save  the  mission  aircraft  from 
Imminent  destruction.  In  some  cases,  only  seconds 
may  be  available  for  the  proper  sequence  of 


recognition/decision/reaction.  Tha  air-to-air 
engagement  of  an  enemy  aircraft  will  almost 
certainly  be  mission  critical  for  a  fighter 
aircraft  with  an  air  offensive/defensive  mission. 
However,  the  same  task  may  not  constitute  a  mission 
critical  task  for  a  ground  attack  aircraft.  These 
are  the  sort  of  considerations  which  must  be 
examined  when  determining  il.e  required  type  of 
training  devices.  The  transition  between  radar 
and  FLIR  sensor  systems  Is  another  task  which  may 
be  deemed  mission  critical.  The  system  operator 
must  correctly  correlate  target  appearance  on  both 
sensors  In  minimum  time,  acquire  and  Identify  the 
proper  target  so  that  a  successful  attack  may  be 
completed.  Again,  there  may  only  be  a  matter  of 
seconds  to  correctly  carry  out  this  task. 

Precision  Guided  Munitions: 

By  Its  very  nature  the  systems  application  area 
of  PGM  training  readily  lends  Itself  to  the  concept 
of  part  task  training.  PGMs  are  add-on  weapon 
systems  that,  although  utilizing  aircraft  systems 
and  data,  and  requiring  varying  degrees  of  hard¬ 
ware  Installation  In  the  parent  aircraft,  may  be 
regarded  as  complete  systems  on  their  own  account. 
Hence,  the  development  of  a  PTT  to  train  PGM 
weapon  systems  readily  (and  even  Intuitively) 
comes  to  mind.  Other  factors  combine  to  strongly 
suggest  the  development  of  a  PTT  to  meet  PGM 
training  requirements.  Many  of  them  utilize  high- 
resolution  Imaging  sensors  for  target  acquisition 
and  homing,  and  have  associated  cockpit  displays 
which  require  the  training  of  operator  visual 
discrimination  skills.  Some,  such  as  the  GBU-15 
and  PAVE  TACK,  require  special  manual  skills  for 
tracking  and  weapon  control .. .skills  which  are 
perishable  and  which  require  continual  training. 
Usually  the  delivery  of  a  PGM  Is  also  e  mission 
critical  task  and  requires  intensive  training  so 
that  successful  delivery  of  the  PGM  Is  virtually 
guaranteed. 

Other  factors  resulting  from  the  Increasing 
proliferation  and  complexity  of  PGMs  and  related 
systems  need  to  be  considered.  Newer  aircraft 
with  digital  avionics  are  being  developed  with 
provision  for  a  range  of  add-on  systems  and  PGMs. 
For  example,  some  tactical  aircraft  will  be  capable 
of  carrying  the  PAVE  TACK  IR  sensor/laser  desig¬ 
nator,  with  some  combination  of  GEU-15,  AGM-65 
Maverick,  and  laser-guided  ordnance.  Each  of  these 
complex  weapon  systems  requires  extensive  system 
knowledge  by  the  aircrew  and  also  requires  consid¬ 
erable  continuation  training  In  the  associated 
cognitive  and  manual  skills.  For  example,  the  HUD 
and  Multi  Function  Display  (MFD)  may  display 
different  symbology  and  different  alphanumerlcs 
for  each  weapon  system.  The  tracking  handle  may 
perform  different  functions  for  different  systems. 
Different  radar  functions  will  be  required  to 
support  different  PGM  operations.  Apart  from  the 
task  of  Individually  learning  to  operate  each 
system,  a  requirement  to  remain  current  In  more 
than  one  system  also  adds  weight  to  a  PTT  approach. 
Ideally,  crews  should  be  designated  for  one  parti¬ 
cular  PGM  system;  however,  this  may  not  always  be 
possible.  If  crews  are  required  to  be  proficient 
in  more  than  one  PGM  system,  careful  thought  will 
have  to  be  given  to  the  training  program  so  that 
all  the  tasks  required  for  one  particular  PGM 
system  are  trained  as  one  training  unit.  This  Is 
to  ensure  that  the  operation  and  Interaction  of 
switches,  function  displays  and  controls  are 
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learned  to  be  associated  with  that  specific  PGM 
system  and  are  not  confused  with  other  functions. 
This  training  of  PGM  systems  as  a  separate  unit 
again  suggests  a  PTT  approach. 

Also,  given  that  the  delivery  of  the  PGM  Is 
normally  a  mission  critical  task,  a  PTT  may  well 
be  appropriate  so  that  extensive  pre-mission 
training  can  be  provided  on  the  PGM  tasks.  Thus, 
the  system  operator  will  be  trained  and  "hot11 
on  PGM  delivery  tasks  prior  to  each  mission.  In 
this  context,  the  possibility  of  crew  room  PTTs 
suggests  Itself  as  means  of  encouraging  aircrews 
to  attain  pre-mission  peaks  In  their  PGM  skills. 

A  light-hearted  extension  of  this  training  system 
envisions  a  scene  where  PTTs,  dressed  up  as 
arcade  machines  with  flashing  lights  and  sound 
effects,  line  the  walls  of  a  crew  ready  room. 
Utilization  would  not  be  a  problem! 

A  major  concern  for  PGMs,  of  course,  lies  In 
the  Individual  weapon  cost.  With  some  weapons 
costing  hundreds  of  thousands  of  dollars,  It 
becomes  a  very  costly  exercise  to  train  regularly 
using  live  weapons.  Dependent  on  relative  cost, 

It  Is  likely  that  a  realistic  PTT  may  be  more  cost 
effective.  However,  there  may  also  be  a 
requirement  for  a  PGM  training  capability  to  be 
integrated  Into  selected  WSTs  for  full-mission 
training.  Depending  on  requirements,  a. range  of 
training  devices  including  WSTs  and  PTTs  may  be 
required.  Some  suggestion  In  regard  to  mixing 
WSTs  and  PTTs  will  be  discussed  later. 

Digital  Avionics: 

A  major  area  for  PTT  application  Is  digital 
avionics  familiarization  In  both  pre-flight 
and  flight  areas.  Many  new  systems  now  require 
significant  data  entry  (way  points,  targets, 
delivery  modes,  etc.)  In  the  pre-takeoff  period, 
and  In-flight  data  alteration  due  to  mission 
changes.  Modern  aircraft,  particularly  tactical 
aircraft,  can  have  several  of  these  sophisticated 
systems  arrayed  at  each  crew  station  (pi  lot,  WSO). 
The  training  problems  Inherent  In  these  systems, 
which  eventually  are  control  centers  for  all 
missions,  are  extensive.  A  complete  understanding 
of  the  operating  modes,  Interactions,  and  anomalies 
of  each  system  will  allow  the  crew  member,  with 
regular  practice,  to  perform  the  assigned  mission 
task  using  alternate  or  backup  modes  as  necessary. 

Single  copies  of  unique  PTTs  for  some  of  these 
avionics  systems  have  been  produced  through  a 
variety  of  means  (organizational,  inter-unit 
projects,  contractual,  research,  etc.).  Often  the 
driver  was  a  desperate  need  from  the  field  for  an 
effective  means  to  allow  operators  to  master  the 
skills  associated  with  new,  unfamiliar,  complex 
systems.  The  effectiveness  of  these  part- task 
devices  has  usually  been  high,  possibly  because 
the  crews  perceived  a  real  need  for  a  trainer,  and 
were  willing  to  live  with  minor  inconveniences  and 
non-relevent  inaccuracies  where  correction  would 
have  increased  trainer  complexity  and  cost.  A 
recent  example  of  such  a  device  Is  the  ARN-101 
Navigation  System  Trainer  prototype  assembled  by 
Ogden  ALC  for  the  Tactical  Air  Command  (TAC).  The 
ALC  personnel,  with  depot  ARN-101  hardware  and 
software  experience,  assembled  an  add-on  crew 
training  device  as  a  peripheral  for  the  existing 
Mission  Data  Transfer  System  (MOTS)  already  used 
by  TAC  aircrews  to  prepare  and  load  ARN-101  way¬ 


point  data  during  pre-mission  ready-room  prepara¬ 
tion.  Since  one  MDTS  will  be  colocated  with  each 
ARN-101  equipped  unit,  and  the  MOTS  computer 
system  hat  spare  computational  capacity  available, 
the  construction  of  an  interface  assembly  to  drive 
a  subset  of  the  aircraft  mission  avionics  "black 
boxes"  and  Indicators  has  produced  a  very  profit¬ 
able  training  device  In  minimum  time  with  little 
additional  new  hardware.  The  prototype  device  has 
received  excellent  reviews  from  users,  who  look 
forward  to  the  completion  of  several  more  of  the 
MDTS  add-on  trainers.  We  must  be  alert  to  these 
opportunities  to  provide  current,  effective 
training  devices,  and  attempt  to  utilize  skilled 
resources  (often  from  existing  programs)  to  assist 
the  timely  development  of  training  equipment.  This 
opportunity  Is  often  neglected  or  ignored  during 
aircraft  hardware  development  and  integration.  By 
such  neglect,  the  operational  user  Is  often  denied 
an  effective  means  of  training  assigned  mission 
tasks  for  the  life  of  the  weapons  system,  and  may 
even  resort  to  sending  aircrews  TDY  to  the  weapons 
system  contractor's  plant  to  train  on  any  available 
device.  Such  an  arrangement  is  not  attractive  to 
the  using  command,  which  experiences  loss  of  air¬ 
crew  availability  and  uncertain  training  effective¬ 
ness,  nor  to  the  weapons  system  contractor  who  Is 
"coerced"  Into  maintaining  a  training  facility... 
often  by  conversion  of  a  portion  of  Internal  R&D 
facilities  and  resources. 

Electronic  Warfare: 

A  third  applications  area  where  PTTs  have 
already  been  applied  with  resounding  success  is  in 
EW  training.  Practical  EW  PTTs  have  ranged  In 
size  from  a  desktop  device  (the  size  of  a  hi-fi 
receiver)  used  principally  for  equipment  familiari¬ 
zation  by  Individual  students,  to  a  multi-station, 
mul tl -computer  driven  electronic  "classroom," 
with  multiple,  individual  student  positions 
(Simulator  for  EW  Training  (SEWT)  (by  AAI)  at 
Mather  AFB  CA).  The  two  basic  simulation  techni¬ 
ques  of  software  modeling  and  equipment  "stimula¬ 
tion"  have  been  used  alone  and  In  many  hybrid 
proportions  to  produce  a  variety  of  EW  training 
equipment.  Some  of  these  PTTs  can  be  (or  are 
presently)  attached  to  a  WST  or  OFr  to  produce  a 
realistic,  correlated  cockpit  EW  display  as  a 
segment  of  an  overall  integrated  training  mission, 
or  used  in  a  "stand-alone"  mode  as  true  PTTs  if 
the  scheduled  WST  training  mission  does  not 
involve  EW  In  the  scenario.  An  example  of  such  a 
device  Is  the  EW  section  of  the  A-10  OFT  (Figure  7). 
Included  with  the  basic  A-10  OFT  configuration  is 
a  dual  section  console  desk/equipment  rack,  with  a 
sliding  partition  between  the  two  console  sections. 
Although  the  computer  which  operates  the  EW 
section  is  one  of  several  that  also  operate  the 
OFT,  and  Is  not  readily  separable  from  the  balance 
of  the  OFT,  ‘he  concept  of  a  "modular,"  multi-use, 
attachable  PIT  is  worthy  of  further  exploitation. 

In  the  example  above,  the  sliding  partition  allows 
student  and  instructor,  seated  side-by-side,  to 
work  closely  together  without  Interphones  or  other 
distractions  for  an  initial  period.  When  student 
evaluation  is  desired,  the  partition  Is  extended 
between  the  student  and  Instructor  positions  so 
that  the  student  is  not  aware  of  Instructor  actions 
except  those  visible  on  the  student  EW  displays. 
Thus,  the  A-10  OFT  (by  Reflectone,  with  EW  by  AAI) 
can  provide  a  separate  EW  part-task  training  mis¬ 
sion  to  another  student  when  the  primary  student  In 
the  OFT  cockpit  does  not  require  EW  displays. 


399 


Figure  7.  A- 1 0  Operational  Flight  Trainer  EW  Auxiliary  Station 


Recently,  a  generic  EW  PTT  has  been  announced 
(Figure  8)  (20).  This  device  Is  reprogrammable , 
through  Its  own  keyboard  (an  option  with  associated 
optional  software),  and  thus  can  be  self-supporting 
at  Individual  sites  for  many  aspects  of  operation. 
If  a  careful  choice  Is  made  for  the  PTT  system 
components  (Computer,  displays,  Interface  equip¬ 
ment,  etc.),  commercial  supportablllty  can  be 
facilitated  for  many  years.  At  the  user  site. 

If  operational  hardware  Is  available,  the  utility 
of  the  device  would  then  be  a  function  of  the 
effort  raised  by  Air  Force  Instructor  personnel, 
many  of  whom  have  proven  to  be  skilled  Innovators 
In  using  less  capable  systems  to  a  greater  degree 
than  the  designers  had  expected, 

EW  PTTs  will  soon  become  airborne  in  an 
innovative  experimental  program  to  provide  In¬ 
flight  training  for  B-52  Electronic  Warfare 
Operators  (EWO)  while  reducing  flight  hours  In 
their  rated  aircraft.  The  Strategic  Air  Command 


(SAC)  has  proposed  the  Companion  Trainer  Aircraft 
(CTA)  program  to  retain  flying  hours  for  B-52 
crew  training  while  lowering  the  operating  cost 
of  the  training  flight  vehicle.  Initially,  two 
T- 39  aircraft  will  be  temporarily  fitted  with 
B-52  crew  training  facilities.  Including  an  EWO 
station  with  a  subset  of  the  EW  systems  found  on 
the  actual  B-52.  The  reduced  size  of  the  CTA 
training  station  Is  primarily  due  to  the  size 
limitations  In  the  T-39  passenger  compartment. 

The  proof-of-concept  EWO  station  will  provide  a 
pre-canned  playback  of  a  pre-determined  mission 
scenario,  with  threat  signals  appearing  at  pre¬ 
scripted  times.  The  student  EWO  actions  will  be 
noted  by  the  controlling  microcomputer  and  logged 
on  a  miniature  paper  tape  printer  for  post¬ 
mission  analysis  and  debriefing.  No  outside 
signals  are  radiated  or  required.  During  Initial 
operational  test  and  evaluation  (IOT&E),  SAC 
will  test  whether  this  simple  PTT  concept  can 
provide  the  required  training. 


Figure  8.  EW  Part-Task  Trainer  for  Individual  Self-Paced  Instruction 
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W*  htve  11  lustre  ted  some  of  the  sreas  of 
modern  digital  weapons  and  avionics  systems  where 
PTTs  can  be  effective  solutions.  The  point  to  be 
made  at  this  stage  is  that  these  types  of  digital 
systems  are  being  developed  at  an  Increasing  rate. 
The  use  of  digital  systems,  micro-miniaturized 
computers  and  technology  advancements,  particularly 
in  the  optical  and  sensor  field  has  facilitated 
the  rapid  development  of  on-board  and  modular 
weapon  systems.  The  adoption  of  a  simulator 
Interface  bus  could  facilitate  Integration  of 
modular  PTT  systems  (n  a  similar  manner  as  the 
MIL-STD-1553  Aircraft  Internal  Time  Division 
Command/Response  Multiplex  Data  Bus  has  assisted 
airframe/subsystem  Integration.  An  added  spur 
to  the  development  and  use  of  modular  systems  is 
the  rapidly  Increasing  cost  of  modern  aircraft. 
With  the  high  cost  of  new  aircraft,  we  are  looking 
to  higher  technology  to  provide  Improved  avionics, 
missiles  and  guided  weapons  to  enhance  the  basic 
parent  aircraft  capability.  A  classic  example  of 
this  process  Is  the  adaptation  of  the  cruise 
missile  to  the  B-52.  For  other  examples,  consider 
some  of  the  modifications  to  our  current  aircraft, 
such  as  the  F-111F,  F-15,  A-10,  F-18,  and  another 
classic  example. . .the  F-16. . .originally  designed 
as  a  lightweight,  visual  fighter.  This  Increasing 
proliferation  of  digital  weapons  affects  all  our 
Armed  Services,  and  unless  we  plan  carefully,  we 
may  well  be  swamped  by  the  training  Impact  of 
these  complex  systems.  Already,  we  are  starting 
to  feel  the  Impact.  Within  the  Deputy  for 
Simulators,  we  have  recently  considered  training 
devices  for  PAVE  TACK,  GBU-15,  AGM-65,  LANTIRN 
and  the  ARN-101  Navigation  System.  The  Navy  and 
Army  and  other  DOD  components  have  obviously  been 
looking  to  similar  training  requirements.  Since 
most  of  these  systems  require  the  type  of  training 
tasks  described  earlier,  which  are  particularly 
applicable  to  PTTs,  we  suggest  that  development  of 
PTTs  to  meet  the  training  requirements  will  be 
effective  solutions  for  cost,  schedule  and 
training  value.  Development  of  PTTs  for  these 
systems  should  allow  for  the  Integration  of  these 
devices  Into  a  WST  to  provide  full  mission 
training.  Ideally,  PTTs  should  be  designed  to 
facilitate  this  Integration  when  and  If  required. 

As  a  first  step,  we  believe  the  Services  and 
Industry  should  collectively  expand  their  studies 
of  unique  training  problems  presented  by  these 
weapon  systems.  Many  of  the  tasks  such  as  the 
visual  discrimination  tasks  apply  across  a  range 
of  aircraft  and  weapon  systems.  There  may  be  some 
general  solutions  where  a  basic  development 
program  may  provide  a  solution  which  Is  useable  on 
different  systems.  As  a  fundamental  approach 
however,  we  believe  that  many  of  these  PTTs  for 
digital  avionics  and  weapons  systems  could  be 
developed  according  to  a  modular  concept  and 
therefore  Interface  requirements  need  to  be  deter¬ 
mined  and  defined  across  the  total  simulation 
spectrum.  "Modular"  and  "standardize"  are  almost 
dirty  words  In  the  simulator  world,  so  let  us 
point  out  that  by  modular  we  do  not  necessarily 
mean  two  units  connected  by  cables.  In  our 
context,  modular  means  a  device  which  can  be 
stand-alone,  but  Is  also  relatively  easy  to  Inte¬ 
grate  Into  a  WST  or  multi-task  trainer  If  required. 
This  concept  for  PTTs  may  require  considerable 
Innovative  engineering  for  some  task  combinations. 
A  modular  concept  for  PTTs  should  provide  a  great 


deal  of  flexibility  In  tha  utilization  of  such 
devices.  As  a  basic  satisfaction  of  training 
requirements,  It  means  that  the  PTT  can  be  used 
as  a  stand-alone  device. .  .perhaps  even  In  a  crew 
ready  room... or  It  can  he  Integrated  Into  a  WST 
at  some  central  location.  Thus,  Initial  and 
qualification  training  can  be  conducted  on  the 
PTT,  full  mission  training  can  be  trained  on  the 
Integrated  WST,  and  continuation  training  of 
perishable  and  manual  skills  can  be  carried  out  on 
the  PTT. 

Another  benefit  of  the  modular  PTT  approach  Is 
that  the  developed  PTT  can  be  utilized  across  the 
range  of  aircraft,  which  utilize  the  weapon  system. 
For  example,  the  A6M-65  Maverick  Missile  Is 
presently  used  by  at  least  seven  aircraft  types. 

A  point  to  be  made  here  Is  that  It  will  not  always 
be  possible  for  the  PTT  to  have  a  common  Interface 
definition  with  a  series  of  WSTs,  particularly 
older  aircraft  WSTs  such  as  the  F-4.  However,  If 
future  PTTs  are  developed  with  common  Interfaces, 
one  side  of  the  Interface  problem  Is  determined, 
and  a  later  upgrade  or  change  of  PTTs  Is  facilita¬ 
ted.  Another  benefit  of  standard  Interfaces  Is 
that  It  may  be  useful  to  combine  certain  PTTs  Into 
a  combined  trainer  to  utilize  a  common  student 
station,  coinnon  data  bases  and  common  display 
equipment  such  as  radar  or  FLIR  displays  (where 
practical).  This  would  be  particularly  useful 
where  the  parent  aircraft  utilizes  both  systems. 

The  flexibility  of  modular  PTTs  should  allow 
them  to  be  relocated  easily  If  a  squadron  or  wing 
moves  or  Is  equipped  with  a  new  weapons  system. 

They  can  also  be  rotated  through  operational  units 
(and  maintenance/upgrade)  on  a  regular  basis.  A 
further  refinement  could  provide  that  the  PTT  be 
designed  so  that  the  various  PTT  components  (con¬ 
trols,  displays,  etc.)  fit  Into  appropriate 
removable  blank  panels  on  a  CFT  or  CPT.  This 
would  provide  an  even  more  realistic  training 
envl  ronment. 

One  approach  to  assist  the  modular/common 
Interface  problem  Is  for  the  WST  and  PTTs  to  be 
designed  to  accept  control  and  Interface  signals 
over  a  standard  simulator  Interface  bus.  This 
could  give  the  simulator  and  PTTs  some  commonality 
with  the  weapons  systems  being  simulated.  The 
parent  simulator  or  PTT  may  not  need  to  have  any 
actual  aircraft  avionics  control  computers  within 
It,  because  MIL-STD-1553  Interfaces  for  several 
types  of  general  purpose  computers  have  already 
been  designed,  tested,  and  Introduced  as  standard 
products  (21,  22).  These  Interfaces  allow  soft¬ 
ware  simulation  of  functional  portions  of  an  active 
multiplex  bus  within  a  general  purpose  computer, 
and  thereby  facilitate  Interconnection  of  varying 
amounts  of  alrciaft  hardware  such  as  HUDs,  air  data 
computers,  stores  management  system  displays,  etc., 
to  a  basic  simulator  or  PTT  (9). 

We  do  not  believe  that  the  full  potential  for 
utilization  of  aircraft  multiplex  bus  equipment  to 
augment  simulator  performance  has  been  examined. 
Avionics  system  design  adherence  to  MIL-STD-1553 
and  other  recently  published  standards  may 
Indirectly  facilitate  the  Interface  of  PTTs  with 
one  another  and  with  WSTs  because  of  the  common 
design  features.  These  standards  are:  MIL-STD- 
1589B  (JOVIAL  J-73  Programming  Language);  MIL-STD- 
1  750A  ( 1 6-B1 1  Computer  Instruction  Set  Architec¬ 
ture);  MIL-STD-1760  (DRAFT)  (Standard  Store 
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Interface:  Alrcra ft/Stores  Elect: feel  Interface 
Definition);  and  ADA  (The  new  DOD  standard 
programming  language  for  command,  control,  and 
avionics  systems)(3).  The  Impact  of  these  stan¬ 
dards  on  simulation  must  be  thoroughly  examined  to 
determine  optimal  design  pol  icies  before  the 
Increasing  development  of  new  weapons  systems  and 
the  resulting  call  for  simulator  upgrades/modifi¬ 
cations  becomes  an  avalanche. 

THE  PAYOFF 

The  success  of  training  devices  Is  usually 
measured  In  terms  of  cost,  schedule,  and  training 
value.  Training  value  has  various  meanings  to 
different  people,  but  generally  will  Include  the 
elements  of  training  effectiveness,  aircrew 
acceptability,  and  Inherent  availability. 
Historically,  PTTs  have  been  successful  In  these 
areas,  probably  because  they  were  generally  less 
complex  and  because  they  were  directed  at  training 
one  particular  task  or  set  of  tasks.  PTTs  of 
modest  scope  and  complexity  have  also  been 
successful  In  terms  of  acquisition  cost  and 
schedule.  For  example,  a  recent  device,  the  F-106 
Aerial  Gunnery  PTT,  was  delivered  2  months  ahead 
of  schedule  and  has  received  favorable  reports 
from  the  users.  In  looking  at  the  Impact  of  an 
Increasing  number  of  complex  weapons  systems  on 
current  and  future  training  requirements,  we  have 
described  some  typical  tasks  and  systems  where  we 
believe  PTTs  will  be  particularly  useful  In  the 
future.  Because  of  the  scope  of  these  applications, 
we  have  also  suggested  a  coordinated  and  planned 
effort  In  the  development  of  these  PTTs. .  .Including 
the  standardization  of  Interfaces  to  facilitate 
Integration  (modularity).  We  believe  the  pursuit 
of  these  approaches  will  provide  opportunities  for 
reducing  cost  and  schedule  during  development  and 
production,  while  meeting  training  objectives. 

For  any  new  program,  the  concept  envisages  a 
spectrum  of  training  devices,  with  lesser  numbers 
of  complex,  costly  WSTs... because  of  the  effective 
use  of  PTTs  to  train  certain  tasks.  Ideally,  the 
PTTs  which  are  developed  for  any  program  would  be 
modular  so  that  they  could  be  readily  Integrated 
with  a  WST  where  required.  WSTs  will  still  be 
required  for  full-mission  training,  but  they  will 
tend  to  be  more  centralized,  with  an  optimum  mix 
of  WST/alrcraft  for  full  mission  training.  The 
thrust  here  Is  that  the  WST  and  aircraft  should  be 
utilized  more  for  full-mission  training  and  evalu¬ 
ation  than  for  Individual  skill  training. 

One  approach  to  simulator  development  which 
has  proven  useful  Is  to  develop  the  simulator 
capability  either  ahead  of,  or  In  parallel  to  the 
aircraft  development  (e.g.,  F-15  program  and  the 
McDonnell  Douglas  F-15  Engineering  Simulator). 
Although  this  concurrency  requires  extra  cost  and 
coordination,  the  simulator  may  primarily  be  used 
as  a  design  tool  or  prototype,  and  can  also  be 
used  as  a  training  device  for  test  pilots  and 
Initial  aircrew  cadre.  The  same  principle  Is  very 
relevant  to  the  development  of  PTTs.  Initial 
front-end  analysis  will  Identify  target  areas  for 
PTTs  (e.g,,H0TAS  and  HUD  operations  and  displays; 

EW  operations;  avionics  operations  and  displays). 
The  prime  aircraft  contractor  could  be  required 
to  build  hot  mock-ups  or  proof-of-concept  devices 
for  these  systems(15).  This  type  of  process  would 
have  several  advantages.  The  tost  devices  are  use¬ 
ful  to  both  the  aircraft  prime  contractor  and  the 


aircraft  system  program  office  as  proof-of-concept 
for  the  new  systems  before  design  freeze.  From 
the  simulator  procurement  point  of  view,  early 
Involvement  of  the  simulator  design  team  with  air¬ 
craft  development  will  produce  a  better,  more 
timely  training  simulation  product.  When  the  air¬ 
craft  design  Is  "frozen,"  experience  gained  with 
the  proof-of-concept  devices  can  then  be  used  as 
the  basis  for  competitive  acquisition  of  the  final 
training  devices  from  Interested  simulation 
contractors.  A  further  advantage  of  this  approach 
Is  that  the  training  effectiveness  of  particular 
PTTs  can  be  assessed  before  comltment  to  a 
production  program.  This  assessment  should 
Include  the  Intended  users  of  the  training  devices, 
and  should  allow  some  re-deflnltlon  of  training 
requirements.  One  result  may  be  suggestions  for 
al  ternatlve' concepts  for  production  PTTs  or  WST 
Integrated  systems,  thus  avoiding  training  devices 
of  low  or  minimal  utility.  Because,  under  this 
concept,  the  Initial  development  work  on  the 
experimental  PTTs  may  be  carried  out  by  the  air¬ 
craft  prime  contractor  (whose  personnel  should 
have  maximum  access  to  development  data),  and 
because  of  the  early  Involvement  of  engineers  and 
others  from  the  government  simulator  procurement 
activity  (to  ensure  proper  documentation  of  para¬ 
meters  used  for  the  PTTs),  cost  and  schedule  risk 
for  the  delivered  production  training  devices 
should  be  reduced.  There  will  be  some  limitations 
with  Isck  of  problem  control  (l.e.,  freeze,  mal¬ 
functions)  on  contractor  R&D  engineering  simulators, 
but  any  action  to  move  simulator  engineers  up  f.om 
the  "tall  end"  of  the  design  data  stream  could 
help  to  shorten  the  multi-year  cycle  of  simulator 
procurement. 

For  systems  such  as  PGMs,  the  cost  and  schedule 
benefits  accrue  from  the  development  of  modular 
PTTs  which  can  be  used  In  several  modes:  as  stand¬ 
alone  devices;  Integrated  as  combined  trainers;  or 
attached  to  WSTs  for  different  aircraft  types. 

These  benefits  are  realized  from  common  PTT  devel¬ 
opment  requirements  and  thereby  a  reduction  of 
future  Integration  effort.  The  modular  approach 
should  also  provide  a  deg  ee  of  flexibility  to 
"swap  out"  PGM  PTTs  when  required.  This  Is  an 
Important  consideration,  as  modern  PGMs  are 
essentially  modular  by  design,  allowing  change  of 
weapons  load  mix  to  suit  Individual  aircraft 
missions  while  preserving  the  precision  guidance 
capability.  PGM  systems  may  also  be  rotated 
among  units  as  the  overall  planning  situation 
dictates,  or  they  may  be  replaced  by  even  newer 
PGM  systems.  The  modular  PTT,  whether  It  Is 
stand-alone  or  Integrated  with  another  device, 
should  be  capable  of  being  swapped  out  with  a 
training  device  for  another  similar  class  weapon. 

Cost  savings  will  also  be  realized  In  the  life 
cycle  of  PTTs  since  considerations  will  Include 
designing  the  training  devices  for  a  specific, 
realistic  life  (maintenance  and  logistics).  Also, 
by  using  engineering  design  guidance,  common  core 
components  can  be  adapted  to  other  requirements  as 
necessary.  Conversely,  If  a  PGM  system  Is  being 
Integrated  Into  an  existing  aircraft,  the  remaining 
service  life  of  the  parent  aircraft  will  obviously 
be  an  early  consideration.  If  the  parent  aircraft 
has  only  a  short  service  lifetime  remaining,  some 
of  the  unique  simulation  tasks  requiring  complex/ 
high  cost  engineering  solutions  will  have  to  be 
Identified  and  consideration  given  to  finding  more 
simplified,  but  effective  PTT  solutions.  In  some 
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cases,  It  might  be  determined  to  be  more  cost- 
effective  to  complete  full-mission  training  In  the 
aircraft. 

Currency  of  operational,  In-use  training 
devices  Is  another  area  for  examination.  Important 
to  the  update  of  PTTs,  as  well  as  other  training 
devices,  Is  a  continuing  analysis  of  training 
requirements  to  determine  modification  needs.  This 
point  Is  frequently  overlooked  and  becomes  the 
loose  thread  that  brings  about  loss  of  confidence 
in  existing  trainers.  When  the  device  no  longer 
matches  the  actual  equipment,  both  students  and 
Instructors  Immediately  recognize  the  difference 
and  soon  resort  to  using  the  actual  equipment  as 
a  trainer.  Since  most  PTTs  can  usually  be  updated 
at  reasonable  cost  (or  In  many  cases  replaced), 
the  problem  Is  less  noticeable  and  causes  less 
loss  of  confidence  In  the  training  system. 

While  we  have  noted  the  rapidly  advancing 
technology  which  Is  producing  modern  weapon 
systems,  we  should  not  forget  that  the  same  tech¬ 
nology  can  also  assist  the  solution  of  our  training 
problems.  We  must  look  for  new  and  Innovative 
applications  of  these  technologies  to  provide  cost- 
effective  training  systems.  For  example,  the 
gamesmanship  designed  Into  current-day  video  and 
computer  games  has  surfaced  new  concepts  In 
presentation  of  data  for  training.  Stress  Is 
increased  by  timing  and  saturation,  and  anxiety  Is 
developed  through  random  presentation  of  unexpected 
challenges.  Such  Ideas  can  meld  the  uses  of  PTTs 
Into  training  syllabll  as  modules  of  training 
events  that  can  be  integrated  Into  broader  training 
tasks  In  the  simulators.  Computer-based  Instruc¬ 
tional  systems  currently  being  acquired  by  the 
services  will  provide  potential  for  management  of 
PTT  resources,  scheduling  them  Into  Currlculums 
and  documenting  their  use.  Such  potential 
heightens  the  applicability  of  small,  microcomputer- 
directed  devices  as  major  components  of  the  overall 
training  program. 

The  students  of  the  1985-2000  time-frame  will 
enter  military  training  with  a  high  level  of 
exposure  to  the  microcomputer  and  game-oriented 
training  from  school  resource  centers.  We  can 
expect  greater  acceptance  of  PTT  concepts  as 
building  blocks  of  Instruction  that  can  task  the 
student  in  highly  stressed  exercises  similar  to 
anxious  moments  of  live  combat.  These  applications 
can  serve  the  transient  needs  of  changing  missions 
and  combat  scenarios  that  now  drive  the  tactical 
and  strategic  training  costs  out  of  sight. 

In  summary,  the  payoff  comes  when  effective 
front-end  analysis  provides  a  dissected  definition 
of  training  requirements  delineating  the  use  of 
various  levels  of  training  equipment.  By 
designing  PTTs  to  build  operator  skill  for  practice 
and  integration  at  the  next  level  of  training, 
Instructors  can  control  training  resources  and 
maximize  the  use  of  moderate-cost  devices  to  bring 
about  a  higher  overall  level  of  skilled  readiness, 
and  thereby  reducing  the  requirement  for  extended 
training  In  the  WSTs  or  the  mission  equipment. 

This  represents  a  change  In  training  strategies 
and  demands  disciplined  instructional  designs,  but 
Is  a  viable  way  to  maintain  high  training  standards 
in  the  future,  and  ensure  better  access  to  current 
training  equipment. 


Standardization  In  engineering  design  and  the 
modularity  concept  should  facilitate  integration 
where  required.  We  have  suggested  that  Innovative 
approaches  In  the  design  and  utilization  of  PTTs 
will  produce  cost  and  schedule  savings  for  a 
particular  set  of  training  tasks.  However,  the 
bottom  line  Is  that  we  must  be  prepared  to  meet 
the  training  requirements  of  an  Increasing  number 
of  complex  and  sophisticated  weapons  systems. 

We  perceive  the  part-task  trainer  as  a  viable 
answer  to  the  dilemma  of  providing  timely,  cost- 
effective  training. 
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ADVANCED  GROUND  MAPPING  RADAR  SYSTEMS 
A  SIMULATION  CHALLENGE  FOR  THE  80's 


John  D.  Stengel ,  Jr. 
Aeronautical  Systems  Division 
Wrlght-Patterson  Air  Force  Base,  Ohio 


ABSTRACT 

The  role  of  the  airborne  ground  mapping  radar  has  been  dramatically  changing  over  the 
last  several  years  due  to  great  technological  strides  In  the  science  of  radar  design. 

Great  improvements  have  been  made  In  both  the  accuracy  and  resolution  of  radar  systems 
primarily  due  to  the  influence  of  high  speed  digital  signal  processing  and  the  development 
of  synthetic  aperature  radar  design.  As  a  result  of  the  many  improvements  to  both  radar 
system  performance  and  flexibility,  the  role  played  by  radar  systems  Is  also  expanding. 

As  advanced  radar  become  part  of  aircraft  avionics  systems,  the  requirement  for  high 
fidelity  training  systems  Immediately  follows.  Changes  to  existing  digital  radar  landmass 
( DRLMS )  specifications  will  therefore  be  required.  In  order  to  meet  the  training  require¬ 
ments  for  advanced  high  resolution  systems  both  enhanced  revisions  of  current  DRLMS  systems 
and  alternative  technologies  may  be  required. 


INTRODUCTION 

Digital  radar  landmass  simulation  (DRLMS) 
technology  developments  over  the  last  ten  years 
have  brought  the  simulated  radar  system  from  not 
much  more  than  a  procedural  type  trainer  to  a 
point  where  world  wide  mission  rehearsal  can  be 
accomplished.  The  current  state-of-the-art  DRLMS 
capability  permits  radar  Imagery  to  be  generated 
that,  in  many  cases.  Is  Indistinguishable  from  the 
actual  aircraft  radar  Image.  Unfortunately  for 
the  simulation  community,  the  state-of-the-art  for 
aircraft  radar  systems  Is  not  static  and  can  be 
expected  to  dramatically  change  over  the  next  ten 
years.  In  response  to  these  changes  to  aircraft 
radar  systems,  significant  changes  and  Improve¬ 
ments  to  DRLMS  technology  will  b?  required. 

Advanced  aircraft  radar  systems  will  become 
increasingly  more  capable  of  generating  high 
resolution  imagery.  This  paper  describes  the 
future  role  of  radar  systems  with  particular 
emphasis  on  the  anticipated  changes  and  additions 
to  DRLMS  specification  requirements  as  a  result 
of  new  aircraft  radar  system,  and  upon  the  poten¬ 
tial  impact  on  DRLMS  design. 

Future  Role  of  Radar 

Future  radar  systems  will  be  called  upon  to 
do  an  increasing  number  of  tasks  as  the  role  of 
combat  aircraft  expands  with  new  mission  require¬ 
ments  while  having  to  contend  with  more  sophisti¬ 
cated  enemy  defensive  capabilities.  The  following 
paragraphs  examine  the  new  roles  and  requirements 
that  are  currently  being  identified,  and  several 
of  the  advanced  capabilities  that  are  being 
developed  to  meet  these  requirements. 

The  trend  for  future  advanced  radar  systems 
nas  already  been  established  by  the  multimode 
capabilities  of  the  FB-111  and  the  advanced  capa¬ 
bility  B-52  radar  systems.  Systems  such  as  these 
employ  not  only  the  basic  ground  mapping  modes 
used  for  general  navigation  but  also  include 
special  spotlight/offcenter  type  high  resolution 


modes  for  weapon  delivery  as  well  as  terrain 
avoidance/terrain  following  modes  to  permit  low 
level  penetration  flight.  However,  performance 
limitations  of  older  multimode  systems  are 
rapidly  being  reduced  on  newer  systems  by  virtue 
of  high  speed  digital  signal  processors  and  on¬ 
board  computer  systems  which  permit  a  great  deal 
of  flexibility  to  be  exercised  In  terms  of  how  a 
radar  signal  can  be  processed,  analyzed,  enhanced, 
and  ultimately  Displayed.  As  a  result  of  the 
enhanced  flexibility  that  is  Inherent  to  the 
digital  signal  processing  capability,  an  advanced 
radar  system  need  no  longer  be  limited  to  the 
basic  capabilities  for  which  the  system  was 
originally  designed.  Both  new  modes  and  enhance¬ 
ments  to  existing  modes  will  be  possible  to  add 
to  advanced  radar  systems  without  major  modifica¬ 
tions  or  redesign.  The  multimode  capability  will 
further  be  enhanced  with  the  inclusion  of  the 
phased  array  antenna.  (1)  Antennas  of  this  type 
will  permit  rapid  changes  from  one  mode  to 
another  and  precise  pointing  at  desired  targets 
by  electronically  steering  the  radar  beam  without 
the  delays  inherent  to  conventional  mechanically 
scanned  antennas.  Examples  of  different  capabil¬ 
ities  that  will  be  a  part  of  an  advanced  multimode 
radar  system  include  conventional  real  beam  ground 
mapping,  enhanced  high  resolution  imagery  using 
doppler  beam  sharpening,  synthetic  aperature 
antenna  with  signal  pulse  compression  techniques, 
terrain  following,  doppler  ground  speed  measure¬ 
ment,  air-to-air  and  air-to-ground  target  identi¬ 
fication,  and  rendezvous  beacon  Interrogation. 

Both  existing  and  future  aircraft  will  be 
called  upon  to  carry  out  an  increasing  number  of 
mission  roles  and  objectives.  Bomber  type  air¬ 
craft  will  be  required  to  accomplish  not  only  the 
more  traditional  low  altitude  penetration  and 
bombing  role  but  also  a  standoff  missile  launching 
role.  Although  accuracy  for  weapon  delivery  has 
always  been  a  goal,  the  requirement  for  high 
accuracy  Is  continually  becoming  more  critical 
due  to  the  decreasing  vulnerability  of  many 
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target  areas.  An  Inertial  navigation  system 
Integrated  with  the  aircraft  weapon  delivery 
computer  Is  capable  of  providing  the  desired 
accuracy,  but  only  when  system  Initialization 
and  periodic  updates  have  been  provided  by  an 
accurate  sensor  source  such  as  radar.  A  synthetic 
aperature  mode  will  be  capable  of  providing  the 
desired  high  resolution  Input  for  both  direct 
weapon  delivery  modes  as  well  as  the  capability 
for  precise  air  launched  missile  programming. 

These  advanced  radar  modes  will  also  permit 
accurate  position  updates  to  be  accomplished  at 
much  greater  ranges  than  with  conventional  radar 
systems.  Accurate  position  updates  will  be 
lilted  only  by  the  radar  llne-of-slght  and  will 
ennance  the  stand-off  capability  and  permit 
greater  flexibility  for  target  aiming  point 
selection. 

Advanced  radar  design  will  be  affected  by 
both  the  natural  and  artificially  Induced  environ¬ 
ment.  A  hostile  enemy  environment  will  place  many 
demands  on  the  radar  system  that  could  not  be 
handled  with  a  conventional  radar.  The  flexibil¬ 
ity  and  inherent  capabilities  of  advanced  radar 
systems  will  permit  a  realtime  analysis  of  the 
electromagnetic  environment  to  be  accomplished 
and  to  initiate  the  appropriate  response.  The 
radar  will  be  required  to  not  only  counter  active 
jamming  and  Interference  techniques  specifically 
directed  toward  the  aircraft  but  also  analyze  the 
environment  and  to  auo1d  detection  from  ever 
taking  place.  The  Inherent  characteristic  of 
microwave  energy  to  penetrate  weather  has  long 
provided  aircraft  with  all  weather,  day/night 
operating  capability.  However,  advanced  radar 
systems  must  also  permit  penetration  of  ground 
foliage.  Built-In  flexibility  will  permit 
advanced  radar  systems  to  keep  pace  with  the 
continuously  changing  environment  of  the  future. 

It  is  readily  apparent,  for  advanced  weapon 
systems  in  general,  that  crew  member  task  loading 
Is  of  vital  concern  due  to  both  the  complexity  of 
the  equipment  and  the  ever  Increasing  number  of 
tasks  that  the  crew  member  Is  called  upon  to  do. 
(2)  As  a  result,  the  design  goal  of  all  advanced 
radar  systems  will  be  to  simplify  the  basic 
operation  and  to  provide  assistance  to  the  crew 
member.  In  general,  the  operation  of  advanced 
radar  systems  will  be  simplified  by  virtue  of  the 
programmable  capabilities  of  both  the  navigation 
computer  system  and  the  radar  digital  signal 
processor.  A  terrain  following  mode  when  tied 
Into  the  aircraft  flight  control  and  navigation 
computer  systems  is  one  existing  way  of  assisting 
the  pilot.  An  example  for  future  radar  systems 
will  be  automatic  target  acquisition  and  classi¬ 
fication  modes  for  both  air-to-ground  strikes 
against  both  stationary  and  moving  targets,  and 
air-to-air  modes  for  enemy  aircraft  Interception. 
The  Inherent  high  resolution  characteristics  of 
synthetic  aperature  radar  (SAR)  and  doppler  beam 
sharpening  (DBS)  modes  when  Integrated  with  the 
accuracy  of  navigation  and  weapon  delivery  compu¬ 
ters  also  reduces  the  crew  members  task  load  by 
providing  an  automatic  means  of  accurately  classi¬ 
fying  a  target  or  aiming  point.  This  capability 
is  essential  In  single  seat  fighter  aircraft  where 
the  workload  Is  high  or  in  any  high  speed  air¬ 
craft  operating  at  low  altitudes  where  the  limited 
radar  range  results  In  a  short  time  for  the  crew 
member  to  react  to  a  target. 


In  summary,  the  role  played  by  advanced  radar 
systems  will  continue  to  Include  the  traditional 
tasks  of  general  navigation  and  weapon  delivery. 
However,  many  enhancements  to  the  basic  capabili¬ 
ties  and  the  addition  of  new  specialized  modes 
will  permit  advanced  radar  systems  to  meet  the 
requirements  for  a  high  performance  sensor  capable 
of  operating  in  a  wide  variety  of  environments  and 
capable  of  reducing  the  crew  member's  workload. 

DRLMS  Requirements  for  Advanced  Radar  Systems 

The  requirements  for  the  simulation  of  advanced 
radar  systems  will  be  based  primarily  upon  current 
DRLMS  specifications.  Although  current  specifica¬ 
tions  have  evolved  since  the  first  DRLMS  specifi¬ 
cation  In  1972,  experience  gained  In  more  recent 
years,  as  well  as  the  more  demanding  requirements 
of  future  radar  systems,  dictates  that  a  number  of 
significant  modifications  be  made.  (3) 

From  the  standpoint  of  the  actual  .  dar  system 
being  simulated,  the  most  obvious  typf^  f  changes 
to  current  simulator  specifications  will  be  the 
Inclusion  of  the  new  types  of  modes  and  require¬ 
ments  for  any  system  unique  characteristics. 
Characteristics  that  may  be  required  Include  such 
Items  as  the  effects  of  radar  platform  motion 
compensation  or  digital  signal  processing  anomolles 
on  the  resulting  radar  imagery.  Analysis  of  a 
specific  radar  system's  operation  will  dictate 
which  unique  system  requirements  will  be  required. 

A  requirements  emphasis  will  be  placed  on  a  DRLMS 
design  approach  that  will  permit  expansion  when 
additional  modes  and  capabilities  are  added  to  the 
aircraft  radar  system.  It  becomes  highly  undesir¬ 
able  for  a  DRLMS  system  to  require  a  total  redesign 
or  possibly  have  to  be  replaced  if  the  existing 
aircraft  radar  Is  modified  with  the  addition  of  a 
new  mode  or  capability. 

Requirements  changes  that  will  most  likely 
affect  DRLMS  processing  capability  include  accuracy 
and  data  density.  In  both  cases,  an  attempt  will 
be  made  to  further  deflnltlze  and  more  quantita¬ 
tively  specify  the  level  of  performance  required. 
DRLMS  accuracy  (processing  and  data  transformation) 
will  continue  to  be  specified  independently  for 
both  terrain  elevation  and  planimetry,  as  well  as 
Independently  for  each  mode  of  operation.  Of  great 
significance,  however,  will  be  tighter  planimetry 
positional  accuracy  requirements  for  the  high 
resolution  modes.  Data  density  processing  require¬ 
ments  will  need  to  be  modified  to  consider  both 
real  beam  conventional  modes  and  synthetic  apera¬ 
ture  high  resolution  modes.  Conventional  real  beam 
mode  density  requirements  will  remain  relatively 
stable;  however,  the  high  resolution  data  density 
processing  requirements  can  be  expected  to  become 
more  stringent.  However,  when  considering  both 
accuracy  and  density  processing  requirements, 
several  Important  factors  must  be  kept  In  mind. 
First,  data  processing  times  for  Individual  radar 
frames  on  actual  aircraft  SAR  modes  are  typically 
several  seconds  long  with  an  update  rate  that  may 
be  even  longer.  Second,  the  Individual  frame 
location  will  most  likely  be  fixed  without  the 
need  to  process  data  In  real  time  as  cross  hairs 
are  slewed  virtually  anywhere  within  the  aircraft 
field  of  view.  Also  of  concern  when  considering 
DRLMS  processing  capability  Is  the  extent  to  which 
world  wide  flight  will  be  required.  Recent 
programs  have  emphasized  the  need  for  mission 
rehearsal  encompassing  potentially  the  entire 
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northern  hemisphere.  This  requirement  will  have 
to  be  reevaluated  in  terms  of  training  utility  and 
effectiveness  -  especially  when  reviewed  In  light 
of  ORLMS  design  complexity  and  demands  that  will 
be  placed  on  high  resolution  data  bases. 

A  goal  for  future  ORLMS  specifications  will  be 
the  addition  of  as  many  quantitative  requirements 
In  place  of  areas  that  are  now  subjectively 
defined.  This  will  be  especially  true  In  the  areas 
of  radar  effects  and  special  effects  where  words 
as  "shall  be  realistically  simulated"  predominate. 
It  Is  also  Important  to  note  that  quantitative 
requirements  will  be  based  upon  training  effective¬ 
ness  studies  whenever  possible.  Since  It  Is 
realized  that  quantitative  performance  characteris¬ 
tics  can  tend  to  drive  both  design  and  production 
costs  of  ORLMS,  careful  attention  will  be  paid  to 
the  desired  fidelity  as  a  function  of  training 
requirements. 

The  DMA  digital  landmass  system  (OLMS)  data 
base  has  been  the  primary  source  of  data  for  the 
simulation  of  conventional  ground  mapping  radar 
systems  over  the  last  ten  years.  (4)  However, 
the  DLMS  data  base  possesses  certain  limitations 
that  will  Influence  both  the  requirements  for 
DRLMS  data  bases  and  the  eventual  design  of 
advanced  DRLMS  systems.  These  limitations  are 
centered  around  the  basic  content  of  DLMS  data. 

DLMS  digital  feature  analysis  data  (DFAD)  Is 
produced  to  a  set  of  resolution  and  feature  type 
Inclusion  criteria  which  results  In  a  chart  like 
representation  of  the  earth's  surface  where  homo¬ 
geneous  type  areas  (e.g.  soil,  trees,  desert,  water, 
etc.)  are  represented;  however,  a  more  precise 
description  of  how  soil  Is  contoured  or  plowed,  or 
how  trees  and  foliage  are  distributed  In  natural 
vegetation  areas  Is  not  contained.  As  a  result. 
Level  1  DFAD,  and  to  a  lesser  degree  Level  2,  will 
be  inadequate  for  meeting  the  data  content  require¬ 
ments  for  a  high  resolution  SAR  mode.  As  Indicated 
in  a  prior  discussion,  areas  of  high  resolution 
Interest  for  a  simulator  training  mission  will  be 
restricted  to  relatively  small  areas  selected 
during  mission  generation.  Various  methods  will, 
therefore,  have  to  be  developed  to  either  enhance 
the  basic  DMA  data  In  these  areas  or  to  provide  an 
alternate  data  source  for  the  high  resolution  modes. 
One  alternative  currently  under  evaluation  Is  the 
use  of  synthetically  enhanced  DMA  data.  (5) 

Synthet  c  breakup  Is  an  automated  process  of 
replaclig  large  homogeneous  DFAD  features  with 
smaller,  more  realistically  sized.  Individual 
features.  (Figure  1)  Although  very  encouraging 
results  have  so  far  been  obtained,  certain  limita¬ 
tions  do  exist.  The  data.,  when  broken  up,  is 
random  In  nature  and  not  necessarily  representative 
of  what  actually  exists  in  the  real  world.  This  Is 
a  result  of  the  limited  number  of  descriptors 
assigned  to  an  Individual  DMA  feature  (e.g.  feature 
type,  surface  material,  predominant  height,  etc.). 
For  example,  for  a  residential  area  digitized  as  a 
single  feature,  the  feature  descriptors  will  not 
provide  any  Indication  of  what  variations,  to  the 
basic  feature  type  exist  In  the  real  world  or  If 
there  are  any  unique  characteristics.  The  capa¬ 
bility  to  generate  more  realistic  synthetic  breakup 
should  be  realized  when  more  descriptive  data  bases, 
such  as  DMAs  experimental  Level  V,  becomes  an  avail¬ 
able  product.  Other  alternatives  to  theneed  for  a 
more  detailed  data  base  are  the  inclusion  of 
generic  models  (l.e.  small  areas  hand  modeled  to 
the  level  of  desired  feature  content  and  fidelity) 


Into  areas  of  high  resolution  Interest  and,  to  go 
a  step  further,  generate  a  complete  generic  data 
base.  Such  a  generic  data  base  might  be  based 
upon  the  DMA  data  base  (e.g.  Level  1,  2,  or  V)  on 
which  extensive  synthetic  enhancement  and  generic 
modeling  is  added  or  might  be  entirely  artificial 
In  content  -  realistic  In  appearance  but  not 
representing  any  real  world  area  -  but  tailored  to 
meet  a  specific  training  need.  In  any  case,  more 
detailed  data  bases  are  the  primary  reason  why 
data  density  processing  requirements  will  be  higher 
for  the  simulation  of  high  resolution  radar  systems. 

Although  not  directly  related  to  the  require¬ 
ments  for  advanced  high  resolution  radar  simula¬ 
tion,  DRLMS  test  requirements  will  also  undergo 
extensive  modifications.  For  the  most  part, 
current  test  requirements  are  general  in  nature 
and  a  great  deal  of  Interpretation  Is  left  to  the 
DRLMS  system  designer.  As  part  of  the  same  effort 
to  quantify  the  requirements  for  DRLMS  fidelity 
and  processing,  DRLMS  test  requirements  will  also 
become  more  quantitative  and  more  definitive  as  to 
how  they  are  to  be  performed.  The  goal  will  be  to 
reduce  the  subjectivity  of  DRLMS  testing  and  to 
achieve  the  most  quantifiable  picture  of  system 
performance  as  possible. 

In  summary,  modifications  and  additions  to 
DRLMS  specifications  for  advanced  DRLMS  systems 
as  described  In  the  preceding  paragraphs  are 
"a  goal"  to  be  accomplished  In  various  states 
during  the  next  five  to  eight  years.  It  Is  hoped 
that  certain  quantitative  performance  and  test 
requirements  can  be  added  In  the  next  two  to  three 
years.  However,  advanced  radar  system  unique 
requirements  will  not  be  available  until  an 
advanced  aircraft  radar  system  Is  fully  defined. 

DRLMS  System  Design  Impact 

One  can  only  speculate  as  to  the  total  impact 
advanced  radar  systems  will  have  on  DRLMS  design. 
After  looking  at  the  anticipated  changes  to  DRLMS 
specifications,  however,  It  does  appear  that  the 
two  primary  areas  of  concern  for  advanced  DRLMS 
will  be  the  data  processing  capability  (i.e.  data 
volume  and  processing  speed)  and  data  base 
adequacy.  It  can  be  anticipated  that  for  conven¬ 
tional  ground  mapping  radar  modes,  current  DRLMS 
technology  will  suffice.  Current  DRLMS  technology 
may  also  serve  as  an  Interim  In  the  near  future 
for  high  resolution  modes;  however,  several 
enhancements  and  several  alternatives  can  be 
considered. 

As  discussed  In  the  previous  section,  synthe¬ 
tic  breakup,  when  fully  exploited.  Is  one  way  of 
enhancing  the  data  base.  One  alternative  to 
breaking  up  the  source  DMA  data  base  would  be  to 
accomplish  the  breakup  of  cultural  features  during 
realtime  processing  thereby  reducing  the  amount  of 
on-line  storage  and  data  base  transformation  time. 
However,  careful  attention  would  have  to  be  given 
to  the  capability  of  generating  a  desired  pattern 
of  artificial  features  and  do  so  in  a  repeatable 
manner  from  scan  to  scan.  Similar  to  the  concept 
of  synthetic  breakup  would  be  the  realtime  genera¬ 
tion  of  texture  to  be  applied  to  terrain  and 
agricultural  areas.  Texture  patterns  should  also 
be  selectable  and  repeatable  but  would  probably  be 
less  critical  than  for  cultural  features.  Generic 
modeling  might  also  be  a  means  of  providing  the 
high  data  content  and  detail  necessary  for  high 
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resolution  modes .  Generic  models  of  small  towns. 
Industrial  complexes,  residential  areas,  etc, 
could  be  hand  modeled  as  part  of  a  generic  library 
collection  to  be  Inserted  In  the  data  base  either 
before  transformation  or  during  realtime  DRLMS 
processing. 

It  appears  that  current  DRLMS  technology  will 
be  adequate  for  conventional  ground  mapping  radar 
modes.  Figure  2  Illustrates  a  comparison  between 
simulated  and  actual  B-52  radar  imagery.  In  order 
to  accomplish  data  retrieval  and  processing  for 
advanced  high  resolution  radar  modes,  however, 
either  a  higher  performance  DRLMS  processor 
(parallel  to  the  conventional  ground  mapping 
channel)  or  an  alternative  high  fidelity  Image 
generation  capability  would  be  required.  This 
would  then  Imply  that  hybrid  DRLMS  systems  may  be 
a  solution  for  the  simulation  of  advanced  multi¬ 
mode  radar  systems.  Computer  Image  generation 
(CIG)  techniques,  used  for  out-the-wlndow  visual 
simulation,  might  be  explored.  This  possibility 
Is  appealing  from  the  standpoint  that  high  resolu¬ 
tion  SAR  modes  are  capable  of  producing  Imagery 
that  In  many  cases  contains  the  same  detail  and 
resolution  as  a  visual  scene.  (Figure  3)  In 
addition,  high  resolution  modes  typically  utilize 
digital  scan  converted  video  which  Is  output  to  a 
raster  scan  type  display.  This  display  format  Is 
also  utilized  for  CIG  Image  display. 

An  alternative  to  be  considered  from  both  a 
data  base  and  processing  standpoint  is  realtime 
processing  of  an  array  of  still  photographs.  (6) 
This  process,  successfully  demonstrated  for  the 
simulation  of  airborne  forward  looking  Infrared 
(FUR)  systems,  also  seems  to  hold  possibilities 
for  high  resolution  radar  modes.  High  resolution 
aircraft  radar  Imagery  (e.g.  synthetic  aperture) 
dopr.'tr  beam  sharpening,  side  looking,  etc.) 
could  be  processed  In  a  similar  manner  as  that  for 
a  forward  looking  Infrared  or  visual  system, 
t*--  ?by  providing  a  simulation  capability  with 

nlly  the  same  level  of  fidelity  as  the 

nal  aircraft  Imagery.  However,  a  significant 
ai  ,t  of  effort  would  have  to  be  devoted  to  the 
qu  ons  of  dynamic  shadow  processing,  dynamic 
sys  .i  controls  Interaction  by  an  operator,  and 
the  processing  of  radar  data  base  photography 
at  very  low  grazing  angles  (l.e.  low  altitudes 
and  Ion'  .anges). 

In  order  to  accommodate  future  growth  of 
DRLMS  systems  as  a  result  of  aircraft  radar  system 
modlt  .Ions,  DRLMS  system  architecture  will  need 
to  be  designed  with  an  understanding  of  what 
potential  changes  may  exist  In  the  future. 

Although  It  Is  certainly  naive  to  think  that  all 
contingencies  can  be  "designed  In"  thereby  elimi¬ 
nating  the  chance  of  DRLMS  system  redesign,  good 
design  planning  should  help  to  Increase  the 
chances  of  a  more  graceful  transition  to  the  new 
capabilities.  For  example,  if  additional  process¬ 
ing  power  Is  required,  what  level  of  effort  would 
be  required  to  expand  with  a  parallel  processor 
or.  If  additional  data  storage  is  required,  what 
level  of  effort  would  be  required  to  add  addi¬ 
tional  memory?  Similar  type  questions  should  be 
asked  of  the  basic  radar  effects.  If  aircraft 
radar  performance  Is  Increased,  how  easily  can  the 
antenna  beam  pattern  or  receiver  effects  be 
modified?  Just  as  modern  digital  radar  systems 
provide  growth  capability  to  aircraft  radar 
systems.  It  is  hoped  that  modern  DRLMS  systems 
will  also  Inherit  this  flexibility. 


SUMMARY. 

During  the  next  ten  years,  we  will  see  a 
quantum  jump  In  the  performance  of  airborne  radar 
systems  as  compared  with  the  last  forty  years. 

As  a  result,  Innovative  new  ideas  will  be  required 
to  advance  the  state-of-the-art  In  DRLMS  system 
technology  In  order  to  meet  the  performance 
requirements  for  advanced  radar  simulation  train¬ 
ing  needs.  It  now  appears  that  either  significant 
Improvements  to  existing  DRLMS  technology  or 
alternative  approaches  resulting  In  hybrid  systems 
will  be  required  as  well  as  a  substantial  increase 
to  the  content  and  detail  of  the  radar  data  base. 
Just  as  a  technical  challenge  existed  In  the 
1970's  to  develop  digital  radar  simulation 
technology  to  meet  the  training  needs  for  conven¬ 
tional  radar  systems,  an  even  greater  challenge 
now  exists  to  meet  the  training  requirements 
expected  for  the  advanced  high  resolution  radar 
systems  of  the  1980's  and  beyond. 
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Figure  3.  Synthetic  Aperture  Radar  Image 
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ABSTRACT 


This  paper  will  present  the  results  of  a  study  which  was  conducted  on  the  R-52/KC-135  Celestial  Training 
Device  (CTD).  The  CTD  Is  really  nothing  more  than  a  moderately  equipped  microcomputer  with  special 
software  which  was  developed  in-house  by  the  Strategic  Air  Command.  Total  cost  of  the  system  Is  less  than 
$5,000.  The  CTD  is  used  by  B-52  and  KC-135  navigators  to  maintain  proficiency  In  celestial  navigation 
skills.  In  the  study  to  be  reported,  twelve  measures  of  navigator  celestial  navigation  performance  were 
examined  before  and  after  the  CTD  training.  Findings  indicated  that  even  with  a  30X  reduction  in 
in-flight  celestial  navigation  training,  performance  on  several  key  variables  showed  significant 
improvement  after  the  CTD  was  delivered.  No  variables  showed  a  decline  In  performance.  In  summary,  the 
findings  supported  the  effectiveness  of  the  CTD  beyond  the  experimenter's  Initial  expectations. 

INTRODUCTION  Headquarters,  Strategic  Air  Command  (SAC),  while 

the  actual  feasibility  of  the  concept  was 
determined  by  the  4235th  Strategic  Training 

The  purpose  of  this  report  is  to  describe  the  Squadron  (STS)  in  August  1978.  The  favorable 

method  and  results  of  a  study  conducted  to  recommendation  from  this  evaluation  resulted  in  a 

determine  the  training  effectiveness  of  the  proposal  for  the  procurement  of  celestial  training 

Celestial  Training  Device  (CTD).  The  CTD  is  now  devices  In  January  1979.  Final  approval  for  the 

being  used  by  all  operational  squadrons  of  B-52s  acquisition  of  the  CTOs  was  given  in  February 

and  KC-135s  within  Strategic  Air  Command  (SAC)  to  1979. 

provide  continuation  training  for  navigators  in 

basic  celestial  navigation.  When  the  CTD  was  The  4235th  STS  was  designated  as  the  command 

introduced  In  late  1979,  aircraft  time  for  program  manager  and  given  the  responsibility  for 

celestial  navigation  training  was  being  program  implementation.  CTD  delivery  started  in 

significantly  reduced.  An  immediate  question  was  August  1979  and  was  completed  by  January  1980. 

whether  this  would  adversely  affect  navigator 

in-flight  performance.  The  study  reported  herein  The  Training  Device 
seeks  to  answer  this  question.  The  next  section 

will  discuss  the  origin,  design,  and  operation  of  As  presently  configured,  the  CTD  is  simply  a 

the  CTD.  The  third  section  will  discuss  the  Northstar  Horizon  micro-  computer  with  a  48K 

training  effectiveness  study  conducted  for  this  primary  memory,  a  5  1/4"  floppy  disk  drive,  an 

device.  internal  clock,  a  printer,  and  the  associated 

software.  A  picture  of  the  CTD  is  presented  in 
BACKGROUND  Figure  1.  The  program  is  written  in  BASIC. 

Essentially  any  microcomputer  with  the  required 

In  early  1978,  two  KC-135  navigators,  peripherals  could  be  programmed  as  a  CTD  with  only 

stationed  with  the  384  AREFW,  McConnell  AFR,  KS,  minor  software  modifications.  The  software  was 

developed  the  concept  and  software  to  perform  designed  to  enable  a  navigator  to  call  up  the 

celestial  training  using  a  micro -computer.  The  computer  program  with  very  simple  commands.  The 

concept  was  approved  for  evaluation  by 
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THE  CELESTIAL  TRAINING  DEVICE 


4] 


FIGURE 


computer  plays  the  part  of  pilot,  aircraft,  and 
celestial  observer. 

The  Training  Profile 

The  training  profile  Is  similar  to  actual 
celestial  navigation  in  the  aircraft  with  some 
exceptions.  The  navigator  Is  in  a  classroom 
environment  that  lacks  the  realism  of  excessive 
noise,  vibrations  and  other  Inconveniences  of 
actual  flying.  Charts,  logs,  reference  books  and 
required  equipment  are  available  to  be  used 
simultaneously  with  the  computer.  The  navigator 
follows  the  mission  In  real  time,  completing  all 
routine  work  required  on  the  charts  and  logs. 
Instead  of  coordinating  with  real  people  In  the 
pilot  and  observer  stations,  he  must  communicate 
with  the  computer  via  the  computer  keyboard. 

Also,  the  computer  has  "built-in"  errors  that  will 
cause  the  navigation  to  be  In  error,  thus 
requiring  continuous  corrections. 

Mission  Summary 

At  the  end  of  the  training  mission,  the 
computer  will  provide  a  print  out  of  the  mission 
actions  called  the  Mission  Summary.  The  Mission 
Summary  provides  aircraft  position  (FIX)  for  the 
following  positions:  (1)  Approximately  every 
five  minutes  of  flight  time,  (2)  At  each  heading 
change,  (3)  When  the  heading  mode  of  operation  is 
changed,  (4)  At  the  start  and  termination  of  the 
navigation  legs,  and  (5)  At  the  mid  time  of  each 
celestial  observation. 

Each  FIX,  with  the  exception  of  number  5 
above,  will  provide  the  time,  geographic 
coordinates,  true  course,  wind  direction  and 
velocity,  drift,  true  heading,  magnetic  or  grid 
heading,  true  airspeed  and  groundspeed. 

The  Celestial  Star  Data  portion  of  the  Mission 
Summary  provides  the  mid -time  of  each  celestial 
observation,  geographic  coordinates,  average  True 
Course  (Track),  average  Ground  Speed  (GS),  sextant 
altitude  observed  (HO),  one  minute  motion 
correction,  True  Zenith  (ZN)  of  body  at  mid -time 
of  the  observation,  and  name  of  body  observed.  If 
the  body  was  not  the  Sun,  Moon,  Mars,  Jupiter, 
Saturn,  Venus,  or  one  of  the  53  navigational 
stars,  the  SHA  and  DEC  of  the  body  will  be  given. 
If  the  navigator  selects  between  more  than  one 
planet  or  star  in  the  field  of  view,  they  would  be 
listed  in  the  Celestial  Star  Data  printout. 

The  final  portion  of  the  analysis  printout 
provides  the  time  logged  on  the  trainer  this 
mission,  time  in  freeze  mode  this  mission,  total 
time  logged  on  diskette  to  date,  and  total  time  in 
freeze  mode  on  the  diskette  to  date. 

Performance  Measurement 

At  the  end  of  the  simulated  celestial 
navigation  mission,  the  student  or  his  Instructor 
can  take  the  Mission  Summary  data  and  replot  the 
flight.  This  repiot  allows  the  student  to  compare 
his  solution  of  the  navigation  mission  with  what 
the  simulated  aircraft,  the  computer,  actually 
flew. 


TRAINING  EFFECTIVENESS  STUDY  DESIGN 

When  the  celestial  training  device  study  was 
conceived  In  January  1980,  most  CTOs  had  already 
been  delivered  to  the  Air  Force  units.  Due  to  the 
shortages  of  flight  crews,  avlatloi  fuel,  and 
heavy  operational  demands,  a  formalized  study  was 
not  considered  practical.  Howe/er,  the  Air  Force 
had  objective  navigator  evaluations  In  the  form  of 
In-flight  performance  skills  on  AF  Form  157  (see 
figure  2).  It  was  then  determined  that  a 
realistic  study  could  be  conducted  by  comparing 
In-flight  navigation  skill  measurements,  as 
recorded  on  the  AF  Form  157s,  before  CTD  practice, 
with  similar  skill  measurements  after  the 
navigators  had  a  chance  to  practice  on  the  CTD. 
Also,  reductions  of  In-flight  celestial  navigation 
training  occurred  after  the  delivery  of  the  CTDs. 

The  plan  called  for  the  flight  record  office 
at  each  base  to  send  copies  of  all  AF  Form  157s 
for  flights  prior  to  the  beginning  of  CTD  training 
to  the  4235  STS.  These  forms  were  reviewed  for 
completeness,  and  If  they  met  predetermined 
criteria,  they  were  used  In  this  study. 

A  single  reviewer  selected  all  forms  to  be 
Included  in  the  CTD  study.  The  predetermined 
selection  guide  "Evaluation  Procedures  for 
Selecting  Qualifying  AF  Forms  157s  for  the  CTD 
Study"  was  used  to  ensure  selection  consistency. 
The  control  data  consisted  of  the  responses  from 
the  19  February  1980  request  for  all  AF  Form  157s 
prior  to  CTD  training.  These  data  yielded  over 
1,000  Form  157s.  Initial  review  eliminated  700  of 
these  that  represented  integrated  flights, 
because  these  flights  allowed  the  navigator  to  use 
additional  navigation  aids  such  as  inertial 
computers.  Since  only  celestial  navigation  data 
Is  presently  available  on  the  Celestial  Training 
Device,  it  was  determined  that  evaluated  flights 
should  be  Subject  to  the  same  restrictions. 

The  300  AF  Form  157s  that  remained  were 
reviewed  according  to  the  "Evaluation  Procedures 
for  Selecting  Qualifying  AF  Form  157s  for  the 
Celestial  Training  Study".  Certain  omissions  of 
less  critical  data  points  were  considered 
acceptable.  The  data  points  considered  to  be  less 
critical  were:  (1)  final  heading,  the  actual 
direction  to  destination  at  the  end  of  the 
mission;  (2)  control  time,  the  predetermined 
estimate  of  arrival  at  destination;  and,  (3)  DR 
Error,  errors  made  in  the  position  plotting  on  the 
chart  that  were  also  included  in  computation  and 
plotting  error.  This  was  In  concurrence  with  4235 
STS  resident  subject  matter  experts.  When  these 
omissions  were  allowed,  121  of  the  Form  157s 
qualified  for  inclusion  in  the  study  and  were  used 
as  control  data. 

On  May  28,  1980,  CTDs  had  been  in  place  long 
enough  to  justify  acquiring  the  remaining  data  for 
the  Celestial  Training  Device  Study.  A  request 
went  to  all  SAC  squadrons  using  a  CTD  to  provide 
the  number  of  basic  celestial  navigation  flights 
flown  from  Oct  1979  to  May  1980.  In  addition, 
they  were  to  send  all  AF  Form  157s  for  basic 
navigation  legs  flown  during  the  months  of  April 
and  May  1980  to  the  4235  STS. 
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AF  FORM  157  figure 


Since  the  operational  units  understood  that 
only  basic  celestial  navigation  flights  were  to  be 
Included,  only  300  forms  were  received.  These 
forms  were  reviewed  under  the  same  conditions  and 
by  the  same  person  as  the  control  data. 

Ninety -nine  of  the  AF  Form  157s  qualified  and  were 
used  as  the  experimental  data  for  our  celestial 
training  device  study. 

Figure  2  presents  a  copy  of  a  completed  AF 
Form  157.  A  total  of  twelve  performance  measures 
can  be  obtained  from  each  form  {l.e.,  for  each 
flight).  These  are  as  follows: 

1.  Final  Heading:  Heading  flown  to 
termination  point  and  used  to  determine  the  final 
DR  position.  Used  to  determine  difference  between 
planned  and  final  TRACK. 

2.  Celestial  Control  Time:  The  difference 
between  the  planned  and  actual  celestial 
observation. 

3.  Percent  Reliability:  Percentage  score 
determined  by  comparing  the  number  of  positions 
scored  within  the  distance  tolerance,  to  the  total 
number  of  positions  scored.  Distance  tolerance 
varies  according  to  type  of  navigation  leg  flown 
and  equipment  authorized. 

4.  Computation  Error:  Error  incurred  In 
celestial  computation  of  an  intercept  of  azimuth 
or  in  pressure  pattern  computation.  Also  Includes 
adjustment  errors  Incurred  as  a  result  of  not 
recomputing  information  when  actual  TRACK  and 
ground  speed  differ  more  than  ten  degrees  or  30 
knots,  respectively,  from  precomputation 
information. 

5.  Plotting  Error:  Error  incurred  as  a 
result  of  misplotting  a  line  of  position  (LOP)  in 
either  range  or  azimuth. 

6.  Dead  Peckoning  (DR)  Error:  Error  incurred 
as  a  result  of  misplotting  a  dead  reckoning  DR 
position,  including  the  final  DR,  in  either  range 
or  azimuth.  An  assumed  position  error  is  also 
considered  as  a  DR  error. 

7.  Computation  and  Plotting  Error:  Distance 
measured  from  replot  final  position  using  correct 
information  computed  and  plotted  cumulatively  from 
the  start  navigation  position  to  the  termination 
estimated  time  of  arrival  (ETA). 

8.  Cumulative  Error:  Distance  between  replot 
final  DR  and  aircraft  scored  position. 

9.  Rating:  Grade  awarded  after  using  the 
number  of  error  points  and  LOPs  accomplished  to 
enter  the  Celestial  Error  Point  Scoring  Chart 
contained  in  SACR  50-4,  Vol  I. 

10.  Major  Error:  Any  error  exceeding  the 
criteria  of  SIGNIFICANT.  Failure  to  plot  an  air 
or  DR  position  for  each  heading  change  of  20 
degrees  or  more. 

11.  Significant  Error:  A  celestial 
computation/plotting  or  any  air/DR/assumed 
position  error  of  more  than  6  but  less  than  15  NM. 
A  celestial/computation/plotting  azimuth  error  of 
more  than  6  but  less  than  10  degrees.  A  wind 
error  more  than  10  but  less  than  30  degrees  and/or 


more  than  8  but  less  than  15  knots. 

12.  Minor  Error:  A  celestial  computation, 
celestial  plotting  or  any  air/DR/assumed  position 
error  of  3  through  6  Hi.  A  celestial  computation 
or  plotting  azimuth  error  of  3  through  6  degrees. 

A  wind  error  of  5  through  10  degrees  and/or  4 
through  8  knots. 

ANALYSIS  AND  RESULTS 

Of  the  twelve  variables  described  earlier,  all 
of  the  variables  except  instructor  ratings  yielded 
interval  data  (l.e.,  data  for  which  an  analysis  of 
group  means  and  variances  would  be  appropriate). 
Instructor  ratings  were  ordinal  in  nature; 
therefore,  means  analyses  would  not  have  been 
appropriate.  Consequently,  chi-square 
goodness -of -fit  tests  were  performed  on  these 
data. 

It  was  proposed  at  the  outset  of  the  study 
that  data  from  B-52s  and  KC-135s  should  be 
combined  into  one  group.  This  was  felt  to  be 
valid  since  there  is  no  logical  reason  to 
differentiate  between  navigators  of  the  two 
aircraft  with  respect  to  the  celestial  navigation 
task.  Procedures  and  training  methods  are 
essentally  identical  for  both  aircraft.  Also, 
some  of  the  Form  157s,  which  were  to  be  included 
in  the  study,  did  not  indicate  which  aircraft  was 
involved.  Therefore,  a  significant  proportion  of 
the  data  would  have  been  excluded  if  it  were 
necessary  to  discriminate  between  aircraft  types 
within  the  data.  To  ensure  the  validity  of  this 
commonality  assumption,  the  data  for  which  the 
aircraft  type  was  known  were  analyzed  for 
potential  differences.  This  ensured  that  no 
performance  differences  existed  which  were 
attributable  strictly  to  aircraft  type.  Tables  1 
and  2  present  these  comparisons  between  the 
pre-CTD  and  post -CTD  groups,  respectively,  on  the 
instructor  ratings. 

Chi-square  goodness -of -fit  tests  conducted  on 
these  data  yielded  values  of  =  .771  (p>.5)  and 
=  .636  (p>.5)  for  the  data  contained  in  tables 
1  and  2,  respectively.  Table  3  presents  this 
information  for  the  cumulative  error  score.  Two 
planned  independent  one-way  ANOVAs  were  conducted 
to  compare  B-52  and  KC-135  performance  on  the  pre 
CTD  data  F (1 , 52 )  *  .0076  (NS);  and  on  the  post 
CTD-DATA  F(1 ,52)  =  .34  (NS).  The  analysis  of 
these  two  critical  measures  of  performance  support 
the  contention  that  B-52  and  KC-135  data  could  be 
combined  for  analysis  on  an  empirical,  as  well  as 
a  logical  basis. 

Table  4  presents  means  of  the  data  comparing 
pre  -  and  post-  CTD  performance  with  respect  to  the 
first  nine  variables.  From  this  table,  we  can  see 
that  significant  improvements  were  observed  for 
Percent  Reliability  and  Cumulative  Error  as  a 
consequence  of  CTD  Training.  No  measures 
indicated  a  deterioration  of  performance.  Table  5 
presents  the  data  for  a  chi-square  goodness -of -fit 
test  for  the  instructor  rating  data.  The  test 
statistic  for  this  table  is  =  9.4  (p  < . 025 ) . 
Table  6  presents  the  data  for  the  major, 
significant,  and  minor  errors  to  determine  whether 
a  shift  had  occurred  in  the  type  of  errors  made 
(i.e.,  relatively  more  minor  and  fewer  major 
errors).  The  statistic  for  this  table  is 
=  3.01  (p  <.25).  However,  also  of  potential 
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Interest  was  whether  the  mean  nunber  of  each  type 
of  errors  made  per  student  had  changed  with  the 
Introduction  of  the  CTO.  Table  7  presents  these 
means  and  Indicates  the  test  statistics  and 
associated  reliability  of  differences  existing 
between  the  means  of  pro -vs.  post-CTD  training 
groups.  It  appears  from  Tables  6  and  7  that  a 
reduction  in  the  number  of  major  errors  may  have 
occurred,  although  the  effect  was  only  significant 
at  the  .25  level. 


i 

Table  1 

Instructor  Ratings  -  B-52 

vs.  KC-135,  Pre  CTD  delivery 

Highly 

Qualified  with 

Qualified  (H) 

Qualified  (Q) 

Training  (T)  UnquaHfled  (U) 

B-52  8  (57. 1) 

4  (28.6) 

1  (7.1)  1  (7.1) 

KC-I3S  38  (70.0) 

8  (20.0) 

2  (6.0)  2  (5.0) 

V* 

P  •  KS 

Table  2 

Instructor  Ratings  -  B-52  vs.  ICC. 135,  Post  CTO  delivery 

Highly 

Qualified  with 

Qualified  (H) 

Qualified  (Q)  Training  (T)  Unqualified 

8-52  22  (71.0) 

7  (22.6)  2  (6.5)  0  (0) 

KC-135  67  (76.0) 

18  (20.5)  3  (3.4)  0  (0) 

.«6 

P  •  NS 

f 

Table  3 

Cumulative  Error  Score  *  8-52  vs.  KC-135 

B-62 

KC-135 

Pre  CTO 

11.5 

11.825 

Post  CTO 

8.19 

8.97 

F(1 ,52)  -  .0076  P  •  fC 

Ml. 52)  -  .34  P  •  NS 

Another  question  which  had  to  be  answered  was 
how  the  nunber  of  in-flight  celestial  navigation 
legs  had  changed  subsequent  to  the  introduction  of 
the  CTD.  The  desirability  of  the  CTO  is  very  much 
a  function  of  htw  large  these  reductions  can  be. 
SACR  51-52  and  SACR  51-135  were  modified  so  that 
the  required  nunber  of  celestial  navigation  legs 
were  reduced.  However,  to  ensure  that  this 
reduction  had  occurred,  data  were  collected 
pertaining  to  the  nunber  of  aircraft  celestial 
navigation  legs  flown  per  month  at  each  base.  Some 
months  had  more  flying  days  available  due  to  the 
nunber  of  weekends  and  holidays.  Consequently, 
these  dates  were  adjusted  to  reflect  the  number  of 
celestial  navigation  legs  flown  per  available 
flying  day. 

Table  8  presents  these  data.  To  summarize, 
there  was  an  average  of  10.17  celestial  navigation 
legs  flown  per  available  flying  day  at  all  of  the 
bases  studied  during  the  months  of  October  and 
November  1979,  when  the  pre -CTD  data  were 
collected.  There  were  an  average  of  7.43 
celestial  navigation  legs  flown  per  available 
flying  day  during  the  months  of  April  and  May 
1980,  when  most  of  the  post -CTO  data  were 


collected,  a  reduction  of  3.27  celestial 
navigation  legs  per  flying  day.  This  represents  a 
reduction  of  30.6  percent.  The  number  of 
navigators  at  these  bases  remained  constant 
thoughout  the  period  of  th  study. 


Table  4 

Pre  vs.  Post  CIO  Celestial  Navigation 

in-flight  Performance 

Me-n  Mean 

Significance 

Measure 

Pre  CTO 

Post  CTD 

df 

Fi.df 

Level 

final  Heading 

6.78 

6.05 

203 

.24 

NS 

Celestial  Control  T ire 

1.66 

1.70 

104 

.01 

NS 

Percent  Rel iabi 1 ity 

97.57 

99.76 

207 

7.56 

<.0l 

Confutation  Error 

.24 

.32 

209 

.84 

NS 

Plotting  Error 

.53 

.62 

209 

.62 

NS 

Dead  Reckoning  Error 

2.07 

2.04 

200 

.007 

NS 

Computation  and  Plotting 
Error 

6.21 

4.50 

207 

1.42 

<.25 

Cumulative  Error 

11.24 

8.79 

209 

4.53 

<■04 

Highly 

Table  5 

Instructor  Rating 

Qualified  with 

Ou.nriei)  (h) 

Qualified  (Q) 

Training  (T)  Unqualified 

total 

Pre  CTO 

63  (68.5) 

18  (19.6) 

4  (4.3)  7(7. 6) 

92 

Post  CTO 

69  (74.7) 

25  (21.0) 

5  (4.2)  0  (0) 

•X*.  9.4 

119 

|  NOTE:  Percentages  indicated 

n  parentheses. 

P  -  <  .025 
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Table  7 

Analysis  of  Hajor,  Significant 
Minor  Error*  mde  per  Flight 

and 

Error  Type 

l^an-per  CtO  Standard  Mean 

Oevlatlon  Post-CTO 

Standard 

Deviation 

t  Significance 
Level 

Major 

Signifies! 

Hi  nor 

,400  .845  .282 
.36/  . 785  .385 
.644  .928  .744 

.585 

.693 

1.16 

1.14 

.17 

.69 

<.25 

NS 

NS 

MONTH 

Table  8 

Number  of  Available  Flying  Days  and 

Celestial  Navigation  Legs  Flown  per  Oay  by  Month 

FIV1NG  DAYS  CELESTIAL  NAV  LEGS  CELESTIAL  NAV  lEGS/OAY 

October  1979 

23 

26? 

11.61 

November 

20 

193 

9.65 

December 

16 

126 

7.88 

J  ittutry  1980 

22 

167 

7.59 

February 

18 

187 

10.39 

March 

21 

143 

6.81 

April 

21 

172 

8.19 

".y 

75 

no 

6.80 

DISCUSSION  AND  CONCLUSION 


There  Is  no  doubt  that  the  data  obtained 
during  this  study  support  the  contention  that  the 
CTD  provides  a  highly  effective  training  medium 
for  the  maintenance  of  celestial  navigation 
skills.  None  of  the  measures  showed  a  significant 
degradation  of  performance  and  several  key 
measures  (percent  reliability,  cumulative  error, 
and  instructor  ratings)  showed  significant 
Improvement  at  the  95%  confidence  level.  Several 
other  measures  (number  of  major  errors  per  flight, 
computation  and  plotting  errors,  and  the 
distribution  of  error  types)  showed  significant 
improvements  at  the  75%  confidence  level.  The 
primary  question  was  whether  the  substitution  of 
the  CTD  for  some  inflight  training  had  caused  a 


deterioration  of  navigator  performance.  The 
answer  is  an  unequivocal  "No."  In  fact,  we  can 
surmise,  with  some  degree  of  confidence,  that 
in-flight  performance  has  in  some  ways  Improved  as 
a  consequence  of  the  CTD. 

Because  a  performance  Improvement  was 
observed,  any  estimate  of  the  Transfer 
Effectiveness  Ratio  (TER)  will  probably 
underestimate  the  value  of  the  CTD.  The  Transfer 
Effectiveness  Ratio,  as  defined  by  Roscoe  (1971), 
provides  a  measure  of  the  relative  training 
effectiveness  of  training  device  practice  as 
compared  to  training  on  the  actual  equipment.  For 
the  purposes  of  this  study,  this  translates  to 
following  formula: 

Number  of  In-fl'ght  celestial  N^iber  of  In-flight  celestial 
navigation  trailing  legs  per  -  navigation  training  tegs  per 
TCR  -  unit  tine  prior  to  CTO  delivery  unit  time  subsequent  to  CTD  delivery 
*  Number  of  CTD  celestial  navigation 

training  legs  per  unit  tine 
subsequent  to  CTO  delivery. 


We  were  unable  to  obtain  precise  data  on  the 
number  of  CTD  celestial  navigation  legs  "flown" 
per  unit  time.  However,  SAC  Regulation  51-52, 
Volume  7  requires  that  each  navigator  fly  a 
minimum  of  one,  and  a  maximum  of  three  per 
quarter.  If  we  use  the  "three  per  quarter  per 
navigator"  figure,  we  will  obtain  a  conservative 
estimate  of  the  transfer  effectiveness  ratio.  To 
allow  common  units,  we  must  correct  the  total 
number  of  navigators  and  days  per  quarter  as  in 
the  following  equation: 

UR* (3  CIO  Mi  I.QS/Qu.rttr /n.vlo.tor  X  300  n.y  tq.tor.j/90  Jj.y./’qu.rt., 

m. 

TER  •  .33 


This  indicates  that  .33  in-flight  celestial 
navigation  legs  are  equal  to  one  CTD  celestial 
navigation  leg  in  terms  of  training  value.  Stated 
another  way,  it  requires  3  CTD  navigation  legs  to 
substitute  for  1  in-flight  celestial  navigation 
leg.  As  was  previously  stated,  however,  this 
probably  underestimates  the  value  of  the  CTD  since 
this  substitution  ratio  appears  to  have  resulted 
in  better  performance.  One  of  the  assumptions  of 
this  formula  is  that  the  pro- and  post -training 
device  groups  are  equal  in  terms  of  skill.  In  our 
sample,  the  post-CTD  group  seems  to  have  performed 
better.  We  can  probably  assume  that  skill 
maintenance  alone  would  require  even  less  CTD  or 
in-flight  training  than  is  presently  required. 
However ,  the  improved  performance  may  be 
considered  to  be  beneficial. 

It  should  also  be  noted  that  this  transfer 
effectiveness  ratio  can  only  be  considered  valid 
within  a  narrew  range  of  utilization  practices. 

If  we  were  to  assume  that  this  held  true  over  all 
levels  of  utilization,  we  could  eliminate  the  need 
entirely  for  in-flight  celestial  navigation  by 
substituting  three  times  the  current  number  of 
in-flight  navigation  legs  with  CTD  navigation 
legs.  We  do  not  propose  this  to  be  the  case,  and 
any  further  reductions  in  in-flight  training 
should  be  carefully  monitored. 
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It  Is  also  Interesting  to  compare  the  transfer 
effectiveness  ratio  to  the  operating  cost  ratio  of 
the  CTO  to  the  B-52  and  KC-135.  This  provides  us 
with  a  measure  of  the  cost  effectiveness  of  the 
CTO.  If  we  consider  the  operating  costs  to 
Include  power  (i.e.,  fuel  and  electricity)  and 
personnel  (I.e. .  0-3  level  navigator)  we  find  the 
following  estimates: 

Po.fr  cost*  liUBItfl  (p*r  hour) 

r-135  (fufl)-UM  Jfllon  JP-48  SI »17/g*l -SI 8?S. 20 
6-52H  <fu«l).25.7S  Jflloo*  JP-4P  11.17/9«l-t3Ql2.75 
CIO  (•I«ctr1c1ty)-.S  KUH  ,ltctnctty»  $.08/K«H*1.04 

Perjonnfl  cost!  (ootK)  Alrcrift  Jfld  CTO 

(«„jw  0-3  Hvtl  from  SACHET } *  $15.33 

Cost  Of  1  hour  KC-135  ■  11825.20*115.53.11840.73 
Cost  of  1  hou-  8-52H  •  S3012.7S-»S15.53-$3023.2S 
Cost  of  1  hour  of  CTO  •  $.08*115. 53*115. 61 

KC-1 35/CT0  Cost  Rotlo  *  KC-135  cost  •  $1840.73  *  117 .9 
CTO  cost  $15.61 

8-52/CTO  Cost  Rftto  •  B-62H  cost  ■  13028,28  •  194.0 
CTb  cost  Sli-61 


What  these  numbers  indicate  is  that,  from  a 
cost-effectiveness  perspective,  if  less  than  117.9 
hours  of  CTD  time  were  required  to  replace  one 
hour  of  KC-135  training,  the  CTO  would  be 
cost-effective  for  KC-135  celestial  navigation 
training.  Similarly,  if  less  than  194  hours  of 
CTD  training  time  provided  the  equivalent  of  one 
hour  of  B-52  H  training,  the  CTD  would  be 
cost-effective  for  B-52H  celestial  navigation 
training.  We  have  already  estimated  that  three 
hours  of  CTD  training  is  equivalent  to  one  hour  of 
aircraft  training.  Therefore  the  CTD  is 
undoubtedly  not  only  a  highly  cost-effective 
training  device,  but  also  provides  an  effective 
training  environment.  Again,  these  data  only 
include  fuel  costs  and  the  navigator's  time  costs. 
To  be  conservative,  the  other  five  crewmembers  in 
the  B-52  and  three  crewmembers  in  the  KC-135  were 
not  included.  Some  may  argue  that  they  are 
receiving  other  effective  training.  Also,  B-52 
fuel  cost  estimates  are  for  the  B-52H  aircraft, 
the  most  economical  model  to  fly.  The  B-52D  and 
R-52G  models  consune  23%  and  13%  more  fuel, 
respectively. 

Recommert  ations 

Based  upon  these  data,  we  have  recommended 
that  SAC  consider  utilization  of  the  CTD  as  an 
effective  means  of  maintaining  celestial 
navigation  skills  for  B-52  art  KC-135  navigators. 

A  substitution  of  three  CTO  navigation  legs  for 
each  in-flight  celestial  navigation  leg  is 
recommended.  At  present,  it  is  proposed  that  a 
minimum  of  two  in-flight  celestial  navigation  legs 
be  required  per  navigator  per  quarter.  However, 
it  is  also  recommended  that  a  further  reduction  of 
in-flight  celestial  navigation  training  be 
considered  on  an  experimental  basis  to  determine 
whether  navigator  proficiency  can  be  maintained 
through  greater  use  of  the  CTO.  The  potential  for 
cost  savings  and/or  better  use  of  available 
in-flight  training  time  is  enormous. 

On  a  more  general  level,  these  data  should 
again  indicate  to  the  training  device  development 
community  that  we  should  seriously  consider  low 
fidelity/low  cost  training  devices.  Too  often  we 


Ignore  simple  solutions  such  as  a  microcomputer 
with  software.  We  are  not  yet  at  the  point  where 
minimum  fidelity  requirements  can  be  specified 
from  a  training  effectiveness  viewpoint.  However 
out  data  base  is  expanding,  and  it  seems  that  less 
physical  fidelity  Is  required  than  we  previously 
expected.  Our  design  philosophy  for  part  task 
trainers  should  be  to  build  the  lowest  fadelity 
device  which  the  user  will  accept.  If  we  do  this, 
we  will  identify  the  lower  hound  of  fidelity  as 
well  as  providing  more  cost  -effective  training 
equi  pment. 
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ABSTRACT 


New  integrated  circuit  signal  processing  devices  Introduced  by  several  manufacturers  are 
usually  considered  to  be  for  specialized  applications.  However,  it  will  be  shown  that  these 
devices  are  "ideally"  suited  for  the  real-time  simulation  of  whole  classes  of  dynamic  subsys¬ 
tems.  Furthermore,  these  devices  could  be  used  to  replace  analog  components  and/or  hardware 
which  are  subject  to  aging,  drifting  and  other  factors  that  result  in  maintenance  problems. 
Since  practically  all  training  devices  involve  placing  a  trainee  in  a  realistic  dynamic  en¬ 
vironment,  these  devices  could  impact  the  training  device  industry  much  like  the  GP  digital 
computer.  The  technological  techniques  required  to  utilize  these  new  signal  processing  chips 
will  be  presented.  Finally,  a  survey  of  training  devices  that  could  be  improved  or  reduced 
in  cost  by  using  these  new  devices  will  be  presented. 


INTRODUCTION 

At  present,  there  are  two  prominent  manufac¬ 
turers  committed  to  capturing  the  emerging  LSI 
(including  single  chip  applications)  signal  pro¬ 
cessing  market.  Their  alternate  approaches  to 
this  market  are  best  presented  in  a  tabular  form 
-  see  Table  1.  In  brief  terms,  one  has  embarked 
on  a  fixed  point  processor  approach,  whereas  the 
other  has  chosen  a  floating  point  processor 
design  approach.  The  fixed  point  approach  is 
easier  to  Implement  in  terms  of  current  technol¬ 
ogy,  and  as  such  is  more  mature  (approximately 
two  years  old).  On  the  other  hand  the  floating 
point  approach  shows  much  more  potential  in 
terms  of  broad-based  applications,  e.g.,  compact 
spectrum  measuring  devices  using  FFT  chips, 
statistical  analysis  devices,  array  processing 
etc.  As  a  bonus,  the  floating  point  processors 


TABLE  1.  FIXED  VS.  FLOAT  IK  FOIKT  FF0CZ5S0F3 


Topics 

fixed  Point 

rks 

Floating  Point 

1.  Hath  Ops 

Fixed  Polnt-soaling 

Floating  Foints-no  mealing 

problws 

problems 

2.  Speed 

Co  roa  ratals 

Comparable 

3.  Design 

Easy  with  single  chip 

Relatively  complex ,  requiring 

and  software  tools 

a  ooaplata  unique  procedure 

for  eeeh  appiioation 

M.  Continuous 

Virtually  Impossible 

Completely  sohlevable  bub 

program- 

(aeaory  not  accessible) 

still  a  rather  difficult 

■ability 

except  for  analoi  control 

design  problea 

of  oenter  frequency,  eto. 

Evolution 

Aether  eeture 

Rather  early  in  development 

phase 

6,  Alts  ms 

Limited  to  dynaalo 

Host  proalas  for  FFT,  statis- 

tivss 

■odels  and  special  non- 

tlcal  analysis,  array  pro- 

linear  subsystems 

oesslng,  etc.  -  but 

aoooapanied  (still)  by 

involved  design  steps. 

do  not  have  the  difficult  and  "pesky"  problems 
associated  with  scaling  fixed-point  processors. 
Several  other  manufacturers  have  committed 
resources  to  enter  this  market.  Still  others 
should  be  considered  as  competitors  by  virtue  of 
their  powerful  general  purpose  microcomputer 
lines  which  can  be  applied  to  varied  and  sundry 
signal  processing  applications. 

When  one  considers  the  millions  of  dollars 
that  manufacturers  have  committed  to  manufactur¬ 
ing  and  marketing  of  newly  emerging  signal  pro¬ 
cessing  chips  (and  chip  families),  one  must  ask 
where  and  how  can  these  technologies  be  applied 
to  trainers.  Hopefully,  some  of  these  questions 
can  be  answered. 

WHAT  ARE  SIGNAL  PROCESSING  CHIPS? 

It  is  unfortunate  that  the  words  "digital 
signal  processing"  or  "Processors"  and  "digital 
filters”  have  become  the  bywords  for  devices 
that  are  more  accurately  described  as  real-time 
digital  dynamic  simulators  or  implementations. 
Traditionally,  managers  and  planners  tend  to 
think  spectrum  shapers  or  band  limiting  devices 
when  referring  to  filters.  Similarly,  signal 
processors  are  thought  of  as  communication  pre- 
and  post-signal  conditioners.  It  is  Important 
to  note  that  these  devices  can  be  used  to  imple¬ 
ment  any  reasonable  size  dynamic  real-time  digi¬ 
tal  model,  such  as: 

1.  Digital  controllers  for  analog  servo- 
systems  . 


Training  seminars  have  been  conducted  on  nation¬ 
al  and  international  levels  for  their  sales/ 
marketing  personnel  -  and  then  followed  up  with 
free  (as  token  costs)  seminars  in  all  areas 
devoted  to  marketing  their  signal  processing 
components. 
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2.  Replace  whole  combinations  of  analog 
devices;  e.g.(  circuits,  spring-mass  systems, 
motor  controllers,  etc. 

3.  Simulate  dynamic  analog  devices  of  any 
type  In  addition  to  Implementing  digital  filters 
and  signal  processors.  In  fact,  these  devices 
show  promise  for  designing  Inexpensive  signal 
generators,  electrical  measuring  devices,  tune¬ 
able  filters,  etc. 

Finally,  It  should  be  noted  that  the  "general 
class"  of  signal  processing  chips  should  Include 
special  purpose  preand  post-filters  to  interface 
junctures,  FFT  spectrum  analysis  chips,  high¬ 
speed  floating  point  multiply/divide  processors, 
etc.  These  specialized  devices  also  play  a  role 
in  processing  signals  although  they  are  not  as 
prevalent  as  dynamic  subsystem  simulators. 

TRAINING  DEVICE  APPLICATIONS 

The  ability  of  this  device  to  simulate  dy¬ 
namic  models  in  software,  to  construct  dynamic 
models  In  hardware  supported  by  software,  and  to 
construct  dynamic  hardware,  has  potential  pay¬ 
offs  in  many  areas  of  training  devices.  As  one 
example  of  each  of  these,  consider: 

A  war  game  simulation  where  many  dynamic 
vehicle  situations  Interact  together  to  pro¬ 
duce  Monte  Carlo  type  results  designed  to 
determine  kill  ratios,  logistics  problems, 
etc.  These  type  problems  are  highly  depen¬ 
dent  on  software  development  and  clever 
model  formulation. 

A  flight  simulator  where  countless  servos 
drive  motion  bases,  visual  optics,  and  con¬ 
trol  loading  devices  for  input  control  such 
as  rudder  pedals.  The  success  of  these  de¬ 
vices  depend  on  implementing  servo  systems 
with  carefully  designed  servo  compensators 
and  controllers  that  permit  the  simulated 
device  and  Its  •'nvironment  to  be  useful  for 
training. 

A  satellite  tracker  trainer  where  dish  drive 
servo  velocity  limits  are  far  below  field 
units  for  safety  purposes.  Then  substantial 
savings  are  realized  by  specially  designing 
small  scale  units  that  behave  like  the  field 
units  in  every  respect  except  for  angle  rate 
limits.  Hardware  redesign  is  therefore 
necessary  to  meet  these  performance  require¬ 
ments  . 

In  view  of  the  processor's  ability  to  be 
substituted  for  the  softw-  interface/hybrid, 
and  hardware  -lei-:  of  t”  simulati  sys¬ 
tems,  its  po;*.'  ':ij.  as  a  .gn  bandwidth  adaptive 
controller  becomes  apparent. 

Simulators  are  naturally  limited  in  system 
performance  when  compared  to  the  real  world  sit¬ 
uations  they  simulate.  The  he  dware  simply  does 
not  have,  and  in  casts  does  .eed  the  range 
and  fidelity  of  the  real  v-  For  example, 
motion  platforms  cannot  ge::  ,ite  the  complete 
range  of  acceleration  cue  characteristics  of  a 
particular  aircraft.  However,  since  these 
requirements  are  task  dependent,  the  system  per¬ 
formance  may  be  optimized  as  a  function  of  task 


cue  requirements  by  control  of  the  system  hard¬ 
ware/software  that  adapts  (or  optimizes)  it  for 
the  task. 

The  processor  can  be  programmed  and  combined 
with  other  chips  to  provide  platform  drive  sig¬ 
nal  shaping  as  a  function  of  specific  input  var¬ 
iables  that  characterize  a  task  (e.g.,  carrier 
landing).  For  example,  it  may  be  desirable  to 
change  motion  platform  gain  as  a  function  of 
slant  range  to  carrier  or  average  stick  input 
deviation  from  trim  to  accentuate  cueing  for 
small  Inputs.  Since  large  Inputs  are  less 
likely  as  the  aircraft  approaches  the  carrier, 
the  platform  cueing  range  is  centered  around  a 
smaller,  localized  aircraft  acceleration  range. 
This  would  be  accomplished  using  the  processor 
by  implementing  filters  in  the  software  and 
modifying  the  filter  gain  and  bandwidth  as  a 
function  of  slant  range. 

A  SPECIFIC  EXAMPLE 

Of  the  many  potential  applications  discussed 
for  use  of  the  "microprocessor  on  a  chip",  let's 
consider  a  typical  state-of-the-art  flight  con¬ 
trols  loading  device.  This  device,  which  is 
used  in  almost  all  flight  simulators  generates 
realistic  control  stick  forces,  with  extensive 
use  of  analog  circuit  cards  for  signal  shaping 
functions.  All-analog  systems  are  generally 
preferred  because  of  bandwidth  requirements  as 
high  as  100  Hertz.  Since  the  fixed  point  type 
of  signal  processor  can  be  operated  at  sampled 
data  rates  of  more  than  10,000,  it  is  Ideally 
suited  for  the  flight  control  loader  devices. 

A  typical  control  loader  is  Illustrated  in 
Figure  1.  Analog  circuit  cards  are  used  for 
stabilization  of  the  control  loops  and  for  gene¬ 
ration  of  the  loader  forcing  functions  such  as 
gradient  and  friction.  Typically,  several  ana¬ 
log  cards  are  required  for  each  control  chan¬ 
nel. 

Figure  2  shows  the  same  loader,  except  with 
the  "processor  on  a  chip"  replacing  the  analog 
cards  and  other  analog  shaping  circuits.  Thus, 
complex  analog  circuitry  with  inherent  aging, 
drift,  and  maintenance  considerations  are  re¬ 
placed  with  the  predictability  and  low  mainten¬ 
ance  qualities  of  a  sampled  data  system. 

In  summary,  requirements  for  dynamic  systems 
and  their  simulations  exist  throughout  the  scope 
of  training  devices.  In  view  of  the  previous 
discussion  "What  are  Signal  Processing  Chips", 
it  must  be  concluded  that  these  (typical  train¬ 
ers  and  training  device)  areas  all  represent  ex¬ 
cellent  candidates  for  signal  processor  chips. 
Furthermore,  the  standard  general  purpose  digi¬ 
tal  advantages  still  apply,  e.g.: 

EPROM  based  dynamic  models  can  be  re¬ 
designed/modified  via  simple  software 
changes. 

Redundant  hardware  simplifies  logistics  and 
maintenance,  not  to  mention  capital  savings. 

System  repeatability  is  assured  for  training 
mods  and  corrections. 


ML 


424 


SIGNAL  PROCESSING  TECHNOLOGIES 

The  technological  background  that  designers 
oust  possess  to  implement  systems  and  subsystem 
designs  based  on  signal  processors  fall  into  two 
basic  categories.  First,  there  is  the  general 
area  of  FFT/Spectrum  Analysis  applications  ap¬ 
plied  to  such  problems  as  Spectrum  Signature 
Analysis  or  System  identification.  These  prob¬ 
lems  require  engineers/designers  trained  in 
noise  and  communication,  FFT  algorithms  formula¬ 
tion,  discrete  and  continuous  signal  apectruoB, 
etc.  At  least  some  graduate  level  work  la 
needed  for  these  type  problems. 

The  second  category  of  dynamic  system  model¬ 
ing  is  typically  unfamiliar  to  most  engineers, 
but  its  level  of  expertise  requirement  makes  it 
easy  to  master,  even  on  a  part-time  basis,  for 
the  dedicated  designer.  The  following  areas  of 
study  with  clarifying  comments  should  serve  to 
place  the  technologies  required  for  signal  pro¬ 
cessor  type  designs  in  perspective. 

1.  Continuous  dynamic  system  modeling  -  an 
area  that  each  engineer  should  possess  by  virtue 
of  his  differential  equations,  circuits,  linear 
controls,  etc.,  background. 

2.  Discrete  dynamic  modeling  -  an  area 
missing  in  many  engineer's  background.  However, 
this  area  is  usually  easily  mastered  by  relating 
to  1.  above  and  studying  difference  equations  - 
transforms  and  discrete  models  from  texts  such 
as  DeRusso,  Roy  and  Close,  Cadzow's  and  Kuo's 
books  on  sample  data  control  systems  (1,4, 5, 6). 

3.  Discrete  and  contir  ous  system  rela¬ 
tions  -  an  area  best  mastered  by  studying 
digital  filters.  Here  the  best  approach  is  to 
review  continuous  filters;  (such  as  Butterworth, 
Chebyshev,  Elliptic,  etc.)  and  then  study  the 
digital  filter  impulse  invariant,  step  invari¬ 
ant,  bilinear  transform,  and  other  methods  of 
digital  filter  designs,  which  are  topics  in 
standard  digital  filter  texts  such  as  Oppen- 
heim's  or  Antinov's  (7,8). 

NOTE:  Again  "digital  filters"  may  be  mislead¬ 
ing.  The  goal  and  result  here  is  to  master  dy¬ 
namic  discrete  modeling  (on  digital  computers  of 
real-world  continuous  dynamic  models  which  rep¬ 
resent  the  environment  of  trainers). 

4.  Digital  hardware  implementation  -  an 
area  almost  all  engineers  have  some  background 
in.  Assuming  areas  1.  through  3.  above  were 
mastered  and  a  discrete  H(Z)  discrete  model  was 
frozen,  then  the  steps  of  design  for  the  chip 
would  be: 

(a)  Code  the  H(Z)  model  in  assembly 
language  (9). 

(b)  Using  a  development  system,  evoke 
the  assembler  and  generate  an  assembly  file. 

(c)  With  the  software,  obtain  an  ob¬ 
ject  file  of  the  H(Z)  model. 

(d)  Evoke  the  simulator  software  and 
verify  the  design  by  input/output  test  function 
runs  (10). 


(a)  Using  the  support  software  and  a 
PROM  burner,  program  the  EPROM  with  the  H(Z) 
object  code. 

(f)  Test  and  evaluate  the  final  de¬ 
sign. 

Thus,  because  of  the  rather  broad  technolog¬ 
ical  background  required  for  signal  processing 
design,  the  area  has  been  slow  to  develop.  Even 
so,  approximately  six  months  of  part-time  study 
with  one  or  two  short  courses  can  bring  most 
engineers  up  to  speed.  This  has  been  typically 
reduced  to  three  months  (one  quarter  course)  for 
those  with  microcomputer  system  design  back¬ 
ground. 

CONCLUSIONS 

It  has  been  postulated  that  signal  process¬ 
ing  LSI  technological  developments  will  Impact 
the  training  device  Industry  primarily  because 
economic  savings  can  be  realized  through: 

1.  Reduced  capital  costs 

2.  Reduced  maintenance  costs 

3.  Reduced  modification  and  review  costs 
(via  software  design) 

Other  benefits  Include: 

4.  Repeatability 

5.  Immunity  to  aging 

6.  Reduced  size,  weight,  accuracy,  etc. 

7.  General  purpose  hardware  -  quantity 
prices. 

In  view  of  these  potential  payoffs,  the  fol¬ 
lowing  conclusions  are  appropriate: 

1.  Steps  should  be  taken  to  utilize  signal 
processor  LSI  technology  throughout  applicable 
trainer  designs.  Considering  the  millions  of 
dollars  committed  by  manufacturers  to  real-time 
Integrated  circuits  signal  processor  design,  it 
is  obvious  that  manufacturer  market  research  has 
identified  a  multl-bljlllon  dollar  application 
area  with  substantial  advancement  in  technology, 
as  well  as  alternate  design  approaches  to  "old" 
problems  with  substantial  economic  savings 
(companies  are  sinking millions  into  hardware 
development  and  marketing).  For  example,  some 
conduct  "running"  nationwide  seminars  and  are 
Introducing  new  chips  and  support  chips.  "In¬ 
side  info”  indicates  "new  version"  Integrated 
circuits  masks  have  gone  to  production. 

2.  Industry,  Academia,  and  Government 
agency  engineers  and  scientists  must  be  educated 
and  prepared  to  take  advantage  of  ~ttie  new  signal 
processing - or  discrete  system  and/or  simula¬ 
tion  design  hardware.  To  do  so  requires  pre¬ 
planning  and  training  of  the  related  work 
forces. 
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3.  Although  the  fixed  point  approach  haa 
encountered  tone  difficulties  with  early  produc¬ 
tion  chips,  it  seems  these  problems  will  subside 
in  view  of  new  designs  on  the  horizon.  The 
maturity  of  this  technology  indicates  immediate 
utilisation, 

4.  The  new  floating  point  processing  sup¬ 
port  chip  families  are  moving  a  little  slower. 
At  this  time  it  is  probably  prudent  to  prepare 
for  their  use  through  training  measures  -  but 
hold  off  a  little  longer  on  real  designs.  Small 
special  purpose  designs  may  be  appropriate  to 
undertake  now. 
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APPENDIX 

In  late  1979  the  electronic  hardware  indus¬ 
try  was  surprised  when  the  first  totally  self- 
contained  single  chip  signal  processor  was  in¬ 
troduced.  The  digital  portion  of  the  processor 
has  architecture  specifically  adapted  for  high 
speed  real-time  signal  processing.  As  an  exam¬ 
ple,  one  of  the  fixed  point  signal  processors 
has  the  following  characteristics: 

4  Analog  inputs  (9  bits) 

8  Analog  outputs  (9  bits) 

Provisions  for  limited  digital  Interface 

192  24  bit  instructions 

Sample  rates  of  12  KHz  (and  even  higher  for 
simpler  models) 

24  bit  fixed  point  Internal  precision 

H(Z)  models  up  to  10th  to  14th  order 

Specialized  instructions  allow  very  short 
simple  program  implementations  of  signal  genera¬ 
tors,  VCOs,  phase-lock  synchronizers,  etc.,  -  a 
multitude  of  applications.  Later  versions  of 
their  architecture  promise  even  more  flexibili¬ 
ties.  Finally,  it  should  be  noted  that  early 
version  chips  still  have  design  bugs  that  should 
subside  with  the  perfection  of  manufacturing 
techniques  (12). 
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Figure  1.  Analog  Based  Loader 


Figure  2.  Signal  Processor  (2920)  Based  Loader 
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C-5A/C-141B  AERIAL  REFUELING  SIMULATOR  TRAINING  EFFECTIVENESS  - 
CONCLUSIONS  FROM  PRACTICAL  EXPERIENCE 


Thomas  E.  Sitter ley,  Ph.O. 
The  Boel ng  Company 
Seattle,  Washington 


ABSTRACT 

Since  1976,  more  than  400  USAF/MAC  pilots  have  started  their  aerial  refueling  training  In  a 
multipurpose  engineering  development  simulator  at  Boeing.  The  Initial  simulator  capability 
provided  B-52  flight  characteristics,  a  generalized  transport  cockpit,  and  a  458v  x  60*h  FOV 
projected  Image  of  a  K-7Q7  Tanker.  The  training  program  was  Improved  through  the  use  of  the 
simulator  from  the  very  beginning.  Higher  student  flying  performance  was  achieved  with 
enhanced  safety.  Within  six  months,  flying  hours  for  training  were  reduced  fifteen  percent. 
Numerous  simulator  modifications  have  been  made  over  the  past  several  years.  Including  Improved 
flight  characteristics,  visual  scene  fidelity,  and  training  features.  This  opportunity  to  train 
operational  pilots  In  an  environment  of  evolving  simulator  configurations  has  afforded  a  new 
understanding  of  the  cost/be  aflt  of  different  levels  of  simulation  sophistication.  Practical 
experience  conclusions  concerning  the  training  effectiveness  of  a  broad  range  of  simulation 
features  are  discussed. 


INTRODUCTION 

To  begin  with,  this  paper  Is  presented  with 
some  trepidation  as  It  could  be  construed  as  taking 
a  step  backwards  In  simulation  technology  rather 
than  moving  forward  In  pushing  the  state  of  the 
art.  Pragmatically,  however,  It  should  be  viewed  as 
a  description  of  Impressions  of  a  highly  cost 
effective  approach  to  an  Immediate  operational 
training  need.  These  Impressions  have  come  from  a 
program  that  grew  by  necessity  and  serendipity 
rather  than  from  plan  and  design.  The  impressions 
and  conclusions  described  are  primarily  subjective, 
based  on  five  years  of  experience  and  observation, 
not  from  the  preferred,  rigorous  scientific  study. 

The  training  of  Military  Airlift  Command  C-5A 
and  last  year,  C-141B,  aircraft  commanders  began  In 
1976  with  a  routine  tour  of  the  Boeing  Visual 
Flight  Simulation  Laboratory  by  the  Commander  of 
MAC.  This  research  and  engineering  facility  had  an 
aerial  refueling  simulation  capability  for  both  the 
receiver  aircraft  pilot  as  well  as  the  tanker  boom 
operator.  The  receiver  pilot  simulation  capability 
was  recognized  as  having  potential  application  to 
an  operational  MAC  training  need. 

The  strategic  concept  of  airlift  operations  Is 
to  deliver  a  payload  anywhere  In  the  world  on  short 
notice.  Conflict  In  the  Mid-East  pointed  out  the 
need  for  enroute  aerial  refueling  of  the  C-5A 
transport  to  achieve  this  objective  (Figure  1).  The 
Immediate  aerial  refueling  qualification  of  a  large 
number  of  pilots,  coupled  with  the  economics  of 
aircraft  flying  hours,  fuel,  and  tanker 
availability,  posed  a  training  problem  of 
significant  proportions. 

There  were  some  strong  technical  questions  on 
the  part  of  both  Boeing  and  MAC  training  personnel 
concerning  the  applicability  of  the  aerial 
refueling  engineering  development  simulation  at 
Boeing,  to  the  training  of  C-5A  pilots.  At  the  time, 
the  simulation  provided  only  B-52  aerodynamics  to 
be  "tiown"  from  a  YC-14  engineering  development 
cockpit.  K-707  and  K-747  tanker  Images  were 
generated  using  T.V.  camera/model  techniques  and 


displayed  with  a  45°v  x  60°h  FOV  by  direct 
projection  on  a  hemispherical  screen  15  feet  In 
front  of  the  cockpit.  The  functional  elements  of 
the  simulator  are  depicted  In  Figure  2.  Obviously 
direct  application  of  the  simulation  situation  to 
C-5A  aerial  refueling  was  open  to  question.  The 
training  need,  however,  transcended  the  questions 
and  the  decision  was  made  to  proceed  with  an 
attempt  to  use  the  simulator  for  training. 


THREEFOLD  TRAINING  PROBLEM:  OVERCOME  ANXIETY, 
RECOGNIZE  VISUAL  CUES,  AND 
DEVELOP  FLIGHT  CONTROL  SKILLS 

The  flying  of  large  transport  aircraft  Is 
relatively  straightforward  -  takeoff,  cllmbout, 
enroute,  approach,  and  land.  Certainly  these  flight 
phases  require  solid  piloting  skills  and 
techniques:  however,  the  old  characterization  that 
transport  flight  encompasses  hours  and  hours  of 
boredom  Interrupted  by  an  occasional  moment  of 
stark  terror  Is  probably  a  reasonable  description. 

Now  enter  the  task  of  aerial  refueling.  Find 
that  fuel-giving  tanker  In  thousands  of  cubic  miles 
of  airspace.  Current  pilot  capabilities  coupled 
with  onboard  navigation  systems  can  put  the  tanker 
In  visual  contact  with  relative  ease.  The  problem 
begins  several  hundred  feet  behind  the  tanker.  The 
closer  you  get  the  harder  It  becomes. 

The  first  training  problem  Is  to  begin  to  move 
the  receiver  aircraft  close  enough  to  the  tanker  to 
make  a  non-destructive  fuel  transferring  contact. 
The  flight  manual  points  out  that  "...flying  two 
airplanes  In  close  vertical  proximity  Is  not  safe." 
All  pilot  training  reinforces  the  emotional  notion 
that  touching  another  aircraft  In-flight  Is 
dangerous.  This  attitude,  coupled  with  the 
transport  pilot's  concept  that  close  formation 
flight  means  thousand  foot  separations,  forms  the 
basis  for  a  strong  fear  of  flying  close.  The  pilot 
must  overcome  this  natural  anxiety  and  realize  that 
he  can  develop  the  close  trail  formation  flying 
skills  necessary  to  safely  perform  the  refueling 
task. 
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Figun  3.  Airitl  R»fu»Hng  Pn-Contnct  Position  Bohind  t  KC-t 35  Tinker 


As  the  pilot  gets  close  to  the  tanker  he  can 
see  a  myriad  of  detailed  features  that  were 
undl scernable  from  a  distance.  These  features 
Include  antennas,  drains,  access  doors,  lights, 
seams,  lumps,  and  bumps.  The  second  training 
problem  Is  to  Identify  and  learn  how  and  when  to 
use  these  tanker  features  In  the  visual  crosscheck 
as  cues  for  detecting  and  maintaining  the  proper 
close  trail  formation  position  behind  the  tanker. 


For  example,  the  receiver  director  lights, 
while  giving  vertical  and  fore/aft  position 
Information,  arc  often  referred  to  as  "too  late 
lights",  telling  the  pilot  where  he  Is  but  not 
where  he  Is  going.  In  daylight,  observation  of  the 
UHF  antenna  as  It  lines  up  with  a  painted  line  on  a 
row  of  rivets  provides  not  only  an  excellent 
vertical  position  cue  but  also  provides  early 
direction  and  rate  of  movement  Information. 
Similarly,  the  KC-135  No.  4  engine  nacelle  can  be 
seen  as  Intersecting  with  th4  C-5A  center 
windscreen  post  to  provide  fore/aft  position  and 
movement  Information  to  the  receiver  pilot. 


Recognizing  that  aerial  refueling  Is  primarily 
a  visual  task.  It  Is  easy  to  understand  that  the 
tanker  aircraft,  framed  In  the  receiver  pilot's 
window,  becomes  simultaneously  the  source  of 
attitude,  altitude,  and  airspeed  information 
(Figure  3).  Where  the  tanker  goes  during  refueling, 
so  must  the  receiver  go.  The  third  training  task, 
then.  Is  to  develop  the  precise  control  skills  and 
responses  to  the  tanker  cues  which  will  maintain 
the  receiver  aircraft  In  the  small  refueling 
envelope  behind  the  tanker. 


Development  of  a  smooth  pilot  technique  is 
necessary  for  safe  and  successful  aerial  refueling. 
Overcontrol  or  rough  usage  of  the  controls  on  the 
part  of  the  pilot  will  cause  a  chain  reaction  with 
progressively  larger  and  larger  corrections 
required  to  maintain  position.  Because  of  the 
magnitude  of  Interrelated  aerotynamlc  effects 
between  the  tanker  and  receiver,  large  and  rapid 
control  movements  may  cause  the  two  aircraft  to  be 
drawn  together  or  thrown  apart,  further  compounding 
the  pilot's  control  problem  and  potentially  causing 
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an  extremely  unsafe  condition.  With  the  great  mass 
and  Inertia  associated  with  transport  refueling, 
the  pilot  must  learn  to  make  very  small  and  very 
early  control  corrections  in  order  to  remain  In 
position  and  to  keep  the  situation  from  getting  out 
of  hand. 

For  initial  aerial  refueling  skill 
acquisition,  the  training  problem,  then,  is 
threefold:  build  confidence  in  the  pilot  that  he 
can  safely  fly  close  enough  to  the  tanker  to 
refuel;  establish  the  meaning  and  use  of  tanker 
cues  for  identification  of  position  and  movement; 
and  refine  the  precision  and  smoothness  of  flight 
control . 

AERIAL  REFUELING  SIMULATION  TRAINING  PROGRAM 
Initial  Feasibility  Test 

Before  committing  to  a  major  simulation 
training  program,  a  few  highly  experienced  C-5A 
aerial  refueling  instructor  pilots  and  flight 
examiners  subjectively  evaluated  the  simulator 
performance.  Based  on  their  recommendations,  the 
next  group  of  C-5A  pilots  scheduled  to  enter  aerial 
refueling  training  were  selected  for  a  simulation 
training  feasibility  test.  Four  pilots  underwent 
training  in  the  simulator  before  receiving  the 
formal  school  training  and  four  pilots  went 
directly  to  the  school.  The  same  instructors  were 
responsible  for  training  both  groups  to 
qualification. 

Comparison  of  the  two  groups'  performance  was 
primarily  subjective  Instructor  assessment 
supported  by  measurement  of  refueling  contact  time 
and  number  of  inadvertant  disconnects  during  the 
training  flights.  The  group  that  had  the 
opportunity  to  train  in  the  simulator  was  judged  by 
the  instructors  to  be  more  stable  on  the  first 
training  flight  and  was  able  to  stay  in  the 
refueling  position  longer  with  fewer  inadvertant 
disconnects.  The  general  assessment  was  that  the 
first  flight  performance  of  simulator  trained 
pilots  was  equal  to  the  second  flight  performance 
of  non- simulator  trained  pilots. 

Reduced  Flying  Training  Program  Est ablished 

I  he  original  C-5A  pilot  aerial  refueling 
qualification  course  at  Altus  AFB  included  ground 
school  instruction,  flight  simulator  training  for 
rendezvous,  aerial  refueling  procedures,  and  crew 


coordination/communications,  six  aerial  refueling 
training  flights,  and  was  completed  v/ith  a  seventh 
flight  for  a  qualification  evaluation.  Based  upon 
the  results  of  the  feasibility  test  a  new  training 
program  was  developed  with  training  in  the 
simulator  substituting  for  the  first  of  the  six 
training  flights.  Each  pilot  would  first  undergo 
aerial  refueling  simulator  training  in  Seattle 
prior  to  reporting  to  the  school  at  Altus  AFB.  At 
Altus  he  would  receive  the  same  training  curriculum 
as  in  the  original  program  but  with  one  less 
training  flight  in  the  aircraft. 

The  simulator  training,  conducted  by  qualified 
USAF  instructor  pilots,  covered  the  essential 
elements  of  the  first  flight:  closure  to 
pre-contact,  recognition  of  tanker  cues, 
demonstration  of  the  refueling  envelope, 
stabilizing  aircraft  movement,  and  closure  to 
contact.  Figure  4  depicts  the  simulated  view  of  the 
tanker  in  the  pre-contact  and  contact  positions. 
Each  pilot  received  approximately  five  hours  of 
simulator  training  spread  over  two  days  to  minimize 
fatigue.  This  training  approach  has  been  followed 
for  the  past  five  years  without  significant  change. 

Evaluation  of  Simulator  Effectiveness 

The  initial  evaluation  of  aerial  refueling 
simulator  training  effectiveness  was  primarily 
subjective,  but  nonetheless,  dramatic.  Prior  to 
using  the  simulator,  apprehension  or  fear, 
undeveloped  pilot  motor  skills,  coupled  with  new 
and  strange  procedures,  all  combined  to  produce 
very  liv.tle  stabilization  behind  the  tanker,  much 
less  any  significant  refueling,  on  the  first 
training  flight.  For  all  practical  purposes,  the 
first  training  flight  was  seldom  productive  in 
terms  of  developing  pilot  technique.  Improvement  in 
confidence,  pilot  skills,  and  technique  were 
immediately  apparent  after  addition  of  the 
simulator  to  the  training  program. 

The  great  interest  generated  by  the  program 
required  that  some  formal  assessment  of  the 
improvement  in  training  effectiveness  be  obtained. 

A  controlled  experiment  using  typical  measures  of 
pilot  performance  to  compare  different  treatment 
groups  was  not  practical .  The  squadron  of 
operational  aircraft  used  for  aerial  refueling 
training  could  not  be  instrumented  to  measure  pilot 
performance  inflight  and  economics  dictated  that 
all  future  pilots  would  be  trained  using  the 
simulator. 


Figure  4.  Simulation  in  Pre-Contact  and  Contact  Positions  as  Viewed  from  the  Right  of  the  Pilot's  Heed 
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Fortunately  there  was  one  practical  and 
meaningful  measure  of  aerial  refueling  performance 
that  had  been  recorded  for  those  pilots  who  had 
previously  undergone  training  without  the  use  of 
the  simulator.  Aerial  refueling  contact  time  (boom 
time)  achieved  by  each  pilot  had  been  recorded  on 
each  training  flight  and  check  ride  by  the  flight 
engineer.  The  same  data  was  recorded  for  each  of 
the  pilots  who  were  trained  during  the  first  5 
months  of  the  new  training  program  using  the 
simulator. 

Pilots  In  the  two  groups  were  all  experienced 
C-5A  aircraft  commanders  undergoing  aerial 
refueling  qualification  training.  The  non-simulator 
trained  group  was  composed  of  56  pilots  who  had 
received  six  training  flights  plus  an  evaluation 
check  flight  for  a  total  seven  mission  syllabus. 

The  simulator  trained  group  was  composed  of  47 
pilots  who  received  five  training  flights  plus  an 
evaluation  check  flight  for  a  total  six  mission 
training  syllabus. 

The  early  subjective  assessment  of  the 
effectiveness  of  the  simulator  was  totally 
supported  by  comparison  of  contact  time 
performance.  On  the  first  aerial  refueling  training 
flight  the  simulator  trained  pilots  achieved  an 
average  of  18.6  minutes  of  contact  time  compared  to 
an  average  of  10.3  minutes  achieved  by  the 
non- simulator  trained  pilots.  This  amounted  to  a 
dramatic  81  percent  Improvement  In  first  flight 
performance.  The  simulator  trained  pilots  continued 
to  achieve  consistently  more  contact  time  on  each 
succeeding  f'ght.  Upon  completion  of  training  with 
the  evaluation  check  ride  the  simulator  trained 
pilots  had  averaged  a  total  of  148.5  minutes  of 
contact  time  during  their  six  aerial  refueling 
missions  In  the  C-5A.  The  non- simulator  trained 
pilots  averaged  149.7  minutes  of  total  contact  time 
during  their  seven  flights. 

Clearly,  use  of  the  aerial  refueling  simulator 
was  shown  to  be  an  effective  substitute  for  one 
aerial  refueling  training  mission.  Furthermore,  It 
was  recognized  that  the  greater  flight  stability 
under  the  tanker  and  fewer  Inadvertent  disconnects 
displayed  by  the  simulator  trained  pilots  directly 
resulted  In  enhanced  flight  safety. 

The  MAC  objective  was  to  train  a  large  number 
of  pilots  per  year,  conserve  aircraft  flying  hours, 
and  reduce  operating  costs;  while  at  the  same  time, 
to  maintain  or  Improve  pilot  proficiency  achieved 
during  training.  These  objectives  have  been 
realized  and  the  cost  benefits  of  the  Improved 
training  effectiveness  afforded  by  the  simulator 
have  been  significant.  Aerial  refueling  training 
sorties  have  been  reduced  by  15  percent,  making 
more  flying  hours  available  for  operational 
missions  and  reducing  the  demand  for  KC-135  tanker 
support.  This  reduction  In  training  flights  for 
more  than  400  pilots  trained  to  date  has  saved  over 
1000  receiver  aircraft  flying  hours.  After 
accounting  for  simulator  and  TOY  costs,  estimated 
current  annual  savings  exceed  $2  million,  or  well 
over  $12,000  per  pilot  trained.  Perhaps  even  more 
significant  Is  the  total  program  savings  of  over  5 
million  gallons  of  fuel. 


OBSERVATIONS  OF  THE  EVOLUTION  OF  THE  SIMULATOR 
CONFIGURATION 

The  original  engineering  simulation 
configuration  for  aerial  refueling  has  undergone 
considerable  modification  over  the  past  five  years. 
Starting  with  a  B-52  receiver  flown  from  a  YC-14 
cockpit  against  K-747  and  K-707  tankers,  the 
simulation  has  evolved  to  aerodynamicaliy  accurate 
C-5A  and  C-141B  receivers  flying  against  a  KC-135 
tanker  with  Interference  flow  field,  down  wash,  and 
bow  wave  effects.  In  approximately  one  year,  the 
YC-14  cockpit  will  be  replaced  with  a  C-141B 
cockpit  In  the  simulator. 

The  opportunity  to  operationally  train  C-5A, 
and  now  C-141B,  pilots  In  an  environment  of 
evolving  simulator  configurations  has  afforded  some 
Interesting  Insights  and  observations  of  the 
cost/benefit  of  different  levels  of  simulation 
sophistication.  What  began  as  a  self  evident  truth 
that  the  fidelity  of  the  training  environment  must 
match  the  operational  world  has  been  opened  to 
question.  While  perhaps  all  flight  simulations  can 
be  opened  to  the  same  questions.  It  Is  Important  to 
recognize  that  the  observations  reported  here  are 
intended  to  be  specific  to  the  aerial  refueling 
training  situation  only. 

Cockpit  Fidelity 

A  relatively  simple  general  purpose 
engineering  development  cockpit  has  been  In  use  for 
aerial  refueling  simulation  research  and  training 
for  the  past  10  years.  Flight  controls,  throttle 
quadrants.  Instruments  and  seats,  not  to  mention 
general  cockpit  size,  configuration,  and  layout, 
have  borne  little  resemblance  to  the  aircraft  being 
simulated  for  aerial  refueling.  However,  It  has 
been  used  successfully  for  training,  starting  with 
B-52  pilots  In  the  early  1970's,  followed  by  747 
cargo/tanker  and  E-4A  pilots,  then  with  C-5A 
aircraft  commanders  continuously  trained  since 
1976,  and  for  the  past  year,  C-141B  aircraft 
commanders. 

An  explanation  of  this  success  may  be  derived 
by  looking  at  the  part- task  nature  of  aerial 
refueling.  Once  visual  contact  with  the  tanker  Is 
made,  the  pilot  gradually  transitions  from  the 
visual  world  of  his  cockpit  and  Instruments  to  a 
focus  on  the  tanker.  By  the  time  the  receiver 
reaches  the  pre-contact  position,  the  pilot's  whole 
visual  world  is  solely  and  completely  the  tanker 
framed  In  his  windscreen.  His  only  relationship  to 
the  cockpit  environment  Is  the  touch  of  the 
controls. 

This  rationale  and  the  actual  operational 
training  experience  to  date  suggests  that  a  single, 
multipurpose  cockpit  configuration  Is  suitable  for 
aerial  refueling  simulation  of  a  number  of  large, 
multi-engine  jet  aircraft.  Specific  cockpit 
Interior  details  and  Instrument  panels  are 
relatively  unimportant.  In  fact.  In  the  current 
simulation  training  program,  flight  Instruments  are 
no  longer  being  used,  having  been  turned  off  by  the 
Instructor  pilots  In  the  first  few  months  as 
unnecessary. 
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The  physical  size,  shape,  and  location  of  the 
flight  controls  Is  probably  of  some  Importance.  The 
controls  In  current  use  are  significantly  different 
than  those  In  the  C-5A  and  C-141B  and  require  the 
pilots  to  spend  a  few  moments  to  adapt  to  the  new 
feel.  While  completely  foreign  controls  In  the 
aerial  refueling  situation  do  not  appear  to  produce 
any  measurable  negative  training,  they  don't  add  to 
the  efficiency  of  the  training.  It  appears 
feasible,  however,  that  a  common  set  of  controls 
with  the  general  physical  feel  and  approximate 
location  can  be  cost  effectively  used  for  multiple 
aircraft  applications. 

Probably  most  Important  Is  the  representation 
of  the  window  size  and  shape.  It  Is  the  frame  of 
reference  through  which  the  tanker  Is  viewed.  Even 
here,  the  precise  replication  of  the  window 
configuration  Is  open  to  question.  The  currently 
used  YC-14  cockpit  window  Is  not  the  same  as  the 
C-5A  or  the  C-141B ,  however  the  top  edge  and  center 
post  are  somewhat  close,  and  do  provide  very 
similar  reference  points  on  the  fuselage  and  No.  4 
engine  of  the  tanker.  It  would  appear  that  as  long 
as  the  window  frame  reference  points  are  close  to 
those  In  the  operational  aircraft,  and  can  be 
appropriately  used  by  the  pilot,  a  single  general 
purpose  window  configuration  can  be 
cost/effectively  accepted  for  aerial  refueling 
training.  For  major  deviations,  a  single 
multipurpose  cockpit  should  still  be  highly 
acceptable  If  equipped  with  replaceable  window 
frames. 

Simulation  Gaming  Area 

The  aerial  refueling  mission  begins  many  miles 
beyond  where  the  tanker  can  be  visually  sighted. 
During  that  time,  rendezvous  procedures  are 
initiated  and  specific  altitude,  airspeed,  and 
distance  control  points  are  achieved.  As  has  been 
previously  stated,  however,  the  significant  part  of 
aerial  refueling  begins  after  the  tanker  comes  Into 
view.  For  cost/effective  training,  then,  the 
auestlon  becomes  how  far  behind  the  tanker  should 
the  simulation  training  begin?  Is  It  at  the  first 
visual  contact,  three  miles,  one-half  mile,  one 
thousand  feet,  or  one  hundred  feet?  Tall  chasing  a 
tanker,  even  In  a  simulator.  Is  not  a  particularly 
effective  use  of  training  time. 

In  the  early  days  of  the  simulation  training 
program  the  instructor  pilots  could  begin  with  the 
receiver  aircraft  several  thousand  feet  behind  the 
tanker.  With  the  exception  of  a  few  "gee-whiz" 
starts  at  this  distance,  the  gap  between  tanker  and 
receiver  was  rapidly  reduced  to  the  no  more  than 
the  300  feet  and  normally  the  150  feet  In  use 
today.  Up  to  that  distance  behind  the  tanker,  even 
the  Inexperienced  refueling  pilot  has  little 
trouble  maintaining  a  controlled  closure. 

The  maximum  vertical  separation  between  tanker 
and  receiver  required  In  the  simulation  was 
similarly  narrowed  very  early  In  the  program  to  no 
more  than  200  feet.  Most  typically  vertical 
separations  have  not  exceeded  100  feet  below  the 
tanker. 


The  lateral  airspace  used  by  the  beginning 
pilot  behind  the  tanker  can  be  relatively  large 
initially,  particularly  as  he  gets  Into  the  usual 
wild  oscillations  from  KC-135  tanker  wing  tip  to 
wing  tip  on  the  first,  flight.  When  this  occurs, 
safe  practice  dictates  that  he  back  away  from  the 
tanker  and  stabilize.  In  the  simulator,  standard 
instructor  pilot  practice  Is  to  reset  to  a 
stabilized  position  behind  the  tanker  whenever  the 
student  flies  beyond  the  KC-135  wing  tips.  This 
eliminates  non-productive  thrashing  about  and  makes 
for  more  efficient  use  of  simulator  time. 

For  all  practical  purposes,  a  realistic  gaming 
area  for  aerial  refueling  simulation  can  be  quite 
limited  while  providing  very  effective  training. 

The  tanker  centered  in  the  front  of  a  gaming  area 
200  feet  wide,  200  feet  high,  and  200  to  500  feet 
deep  can  open  the  doors  to  a  broad  spectrum  of 
cost/effective  visual  simulation  approaches. 

Aerodynamlcs/Handllng  Qualities 

The  precise  pilot  control  techniques  that  must 
be  developed  for  aerial  refueling  would  Intuitively 
dictate  that  the  simulated  receiver  aircraft 
aerodynamics  and  handling  qualities  along  with  the 
Interference  flow  field  and  bow  wave  effects  should 
be  completely  and  accurately  modeled.  Recognition 
of  control  lag  and  correction  of  under  and  over- 
control  are  key  elements  in  aerial  refueling 
training,  once  again,  a  question  may  be  asked:  how 
accurately  must  the  flight  characteristics  be 
modeled  for  effective  training,  and,  at  what  cost? 

Refueling  simulation  training  for  the  C-5A 
began  using  relatively  simply  modeled  B-5? 
aerodynamics,  control,  propulsion,  and 
environmental  effects  models.  The  handling 
characteristics  were  nothing  like  the  C-5A,  but, 
during  the  fi>st  six  months  of  the  program  the 
pilots  were  able  to  fly  the  simulation,  learn,  and 
transfer  their  newly  acquired  skills  to  the  C-5A  In 
flight.  The  often  heard  words  "It  doesn't  fly 
anything  like  the  C-5A"  did  not  prevent  the 
successful  development  of  the  piloting  technique 
that  permitted  the  reduction  of  one  training  flight 
from  the  syllabus. 

Over  the  next  several  years  the  B-52 
characteristics  were  modified  to  somewhat 
approximate  the  C-5A,  particularly  In  roll  and 
power  response.  This  was  accomplished  by  the  time 
honored  tradition  of  "tweaking'  the  mathematical 
coefficients  based  on  experienced  pilot  comments. 
While  the  handling  characteristics  were  stlli  not 
the  same  as  the  C-5A,  the  pilots  were  more 
confident  In  the  simulation.  Perhaps  this  attitude 
was  the  most  Important  aspect  of  the  modification 
because  there  did  not  appear  to  be  any  dlscernable 
effect  on  training  performance. 

When  the  requirement  to  train  C-141B  pilots  In 
aerial  refueling  was  established,  a  complete  and 
proper  modeling  of  the  receiver  aircraft  seemed 
appropriate.  Extensive  aircraft  design  and  test 
data  were  obtained  and  detailed  aerodynamic,  flight 
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control,  and  propulsion  models  Mere  developed  to 
produce  an  excellent  simulation  of  the  C-141B.  This 
new  simulation  model  permitted  refueling  at 
variable  altitudes,  gross  weights,  and  c.g.'s,  and 
Included  effects  of  tanker  downwash, 
tanker/receiver  Interference  flow  field  and  bow 
wave  effects.  Upon  completion  of  the  C-141B 
simulation,  a  similar  effort  was  accomplished  for 
the  C-5A. 

Instructor  and  student  pilot  response  to  the 
new  aircraft  models  was  Immediate  and  excellent. 
They  had  a  simulation  that  was  easily  recognized  as 
“their"  airplane.  Interestingly,  however,  when  In 
the  past  the  pilots  knew  they  were  not  flying  a 
simulation  of  their  airplane,  they  were  extremely 
tolerant  of  gross  differences  In  handling  qualities 
between  simulator  and  airplane.  With  the  new 
simulation  models,  the  most  Insignificant  variance 
was  brought  Into  sharp  focus. 

Based  on  past  observations,  there  was  no 
expectation  that  the  new  simulation  models  would 
result  In  a  dramatic  Improvement  In  effectiveness 
for  initial  qualification  training.  With  only 
limited  training  experience  to  date  on  the  new 
models,  that  appears  to  be  the  case.  There  are  some 
trim  and  handling  quality  improvements;  however, 
that  might  prove  very  significant  If  future 
simulator  continuation  training  Is  considered  to 
maintain  experienced  pilot  proficiency. 

Obviously  the  goal  should  be  to  produce  the 
best  representation  of  the  aircraft  possible,  not 
only  for  pilot  acceptance  but  to  ensure  training 
effectiveness.  This  refueling  simulation  experience 
does  suggest,  however,  that  a  generic  model  for 
large,  multi-engine  jet  aircraft  may  be  an 
acceptable  and  cost/effective  approach  for 
part- task  training. 

Instructional  Features 

Since  the  aerial  refueling  simulation  was 
implemented  in  an  engineering  development  simulator 
facility,  virtually  unlimited  control  over 
simulation  parameters  and  pilot  performance 
measurement  was  Inherently  available.  Any  aircraft 
position,  attitude,  or  speed  behind  the  tanker 
could  be  selected  by  the  Instructor,  and 
aerodynamic  effects  could  be  selectively  added  or 
deleted.  The  Instructor  could  start,  freeze/hold, 
and  stop/reset  the  simulation  at.  will  as  well  as  to 
record  and  playback  any  simulation  flight.  Typical 
performance  measurement  Included  both  analog  and 
digital  recording  of  all  aircraft  flight  parameters 
and  control  Inputs  on  a  continuous  or  instantaneous 
basis,  In  raw  data  form,  Integrated,  or 
statistically  reported. 

Most  of  this  extensive  capability  has  been 
paired  back  over  the  years  to  a  level  of  useful 
practicality.  Selective  addition  and  deletion  of 
aerodynamics  effects  and  aircraft  characteristics 
have  been  found  helpful  In  building  the 
Inexperienced  pilot's  confidence  and  sxlll  as  he 
progresses  through  the  learning  process. 

Instructors  now  use  the  record/playback  feature 
solely  for  an  Initial  demonstration  of  what  closure 
to  contact  should  look  like  and  how  to  recognize 
the  limits  of  the  refueling  envelope.  Occasionally, 
instructors  will  select  a  new  starting  position 
behind  the  tanker,  most  often  to  a  stabilized 
pre-contact  position  which  eliminates 


non-productive  tall  chasing  as  the  pilots  become 
more  proficient. 

Of  all  the  possible  performance  measures  only 
a  few  are  currently  being  recorded  by  the 
Instructors.  Periodically  the  students  fly  a  ten 
minute  evaluation  flight  with  total  contact  time 
and  number  of  Inadvertent  disconnects  recorded. 
This  Is  practical  and  meaningful  data  that  can  be 
used  by  the  Instructor  to  monitor  progress  just  as 
Is  done  on  the  actual  aircraft  refueling  mission. 
In  addition,  It  Is  more  economical  In  terms  of 
reducing  performance  measurement  and  recording 
equipment  and  eliminating  recordings  of  data  that 
were  never  used. 


Visual  Image  Quality  and  Field  of  View 

One  of  the  most  significant  elements  In  aerial 
refueling  Is  the  pilot's  view  of  the  tanker 
aircraft.  The  tanker  becomes  the  receiver  pilot's 
total  visual  world  during  his  approach  to  the 
contact  position.  The  pilot  must  learn  what  the 
tanker  looks  like,  and  how  to  use  the  tanker 
features  as  cues  to  judge  distance,  movement,  and 
rate  of  change.  This  highly  visual  nature  of  the 
aerial  refueling  task  places  great  demands  on  the 
quality  of  the  visual  simulation. 

Pilots  had  great  difficulty  In  achieving  and 
maintaining  a  contact  position  during  some  of  the 
very  early  aerial  refueling  simulation  research  and 
development  at  Boeing.  Even  though  the  physical 
models  of  the  tanker  aircraft  were  accurately 
shaped.  It  became  rapidly  apparent  that  the  visual 
cues  were  Insufficient.  Prominent  features  such  as 
doors,  control  surfaces,  and  windows  were  added  to 
the  tanker  models.  These  features  made  the 
simulated  refueling  task  easier  to  perform. 
Additional  features  such  as  antennas,  drains,  and 
seam  lines  were  then  added  with  a  further  apparent 
Improvement  In  pilot  performance  under  the  tanker. 
While  no  objective  performance  measurements  were 
obtained.  Improvement  In  pilot  performance  was 
obvious.  By  the  time  the  C-5A  simulation  training 
began,  all  that  remained  to  be  added  was  the  row  of 
rivets  In  front  of  the  VHF  antenna,  a  cue 
specifically  taught  In  the  operational  training 
program. 

The  Importance  of  adequate  tanker  feature 
representation  for  aerial  refueling  simulation, 
though  subjective,  was  none-the-less  dramatic.  The 
task  could  be  accomplished  only  with  difficulty 
without  the  presence  of  these  tanker  features. 

After  the  C-5A  simulation  training  began,  the  K-707 
tanker  was  replaced  by  a  KC-135  tanker  without  the 
fan  engines  and  with  longer  engine  struts,  but  with 
all  the  other  tanker  features  essentially  the  same 
(Figure  4).  This  change  In  tanker  configuration 
resulted  in  no  noticeable  difference  In  pilot 
performance. 

It  would  appear  that  It  Is  the  tanker  feature 
detail  that  provides  the  distance,  movement,  and 
rate  of  change  cues  to  the  pilot.  A  conclusion 
based  on  practical  experience  and  observation  Is 
that  aerial  refueling  simulation  Image  quality 
should  be  expressed  in  terms  of  the  representation 
of  tanker  features  that  are  consciously  or 
unconsciously  used  as  perceptual  cues  by  the  aerial 
refueling  pilot. 
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Figure  5.  KC-135  Tinker  Imege  Provided  by  Various  Simulator  Display  Fields  of  View 


Any  discussion  of  Image  quality  must  logically 
Include  an  assessment  of  the  field  of  view  of  the 
tanker  Image  In  which  the  cues  are  represented. 
Obviously,  the  quality  of  the  visual  simulation  Is 
Impaired  If  Important  perceptual  cues  fall  outside 
of  the  simulated  field  of  view.  The  portion  of  the 
tanker  Image  avallaole  to  the  pilot  In  the  contact 
position  Is  depicted  In  Figure  5  for  several 
display  fields  of  view.  Some  portions  of  the  Image 
will  be  occluded  by  the  cockpit  window  frame  and  of 
course  the  boom  wou'M  not  be  seer  as  the  pilot's 
head  Is  forward  of  the  nozzle  when  In  the  contact 
position. 

As  can  be  seen  In  Figure  5,  the  horizontal 
dimension  Is  effected  most  by  the  different  fields 
of  view.  The  C-5A  and  C-14.1B  simulation  training 
program  has  used  both  the  120°  horizontal  ( h)  FOV 
and  the  currently  used  60°h  FOV.  Other  refueling 
simulations  have  used  a  48°h  F0V. 

Analytically,  one  can  show  that  a  field  of 
view  which  Includes  the  tankor  aircraft  wlngtlps  Is 
most  desirable.  An  uncorrected  difference  In  bank 
angle  between  the  tanker  and  receiver  will  cause 
divergent  flight  paths  and  eventual  loss  of  the 
contact  position.  The  receiver  pilot  must  be  able 
to  rapidly  detect  the  onset  and  direction  of  the 
apparent  roll  motion  In  order  to  make  the  early 
aileron  corrections  necessary  to  maintain  a  stable 
position  under  the  tanker.  When  a  divergence  In 
bank  angle  occurs,  the  physical  displacement  of  the 
wing  at  the  outboard  engine  Is  approximately  70 
percent  of  that  at  the  wing  tip.  At  the  Inboard 
engine  the  displacement  Is  about  40  percent  of  that 
at  the  wing  tip. 


Not  only  would  the  larger  displacements  be 
more  rapidly  detected  by  the  eye,  but  vision 
research  has  found  that  short  duration  (less  than  1 
second)  motion  sensitivity  Is  greater  and  longer 
duration  motion  velocity  thresholds  are  achieved 
more  rapidly  In  the  periphery  of  the  visual  field. 
This  means  that  even  while  the  pilot  focuses  on 
details  along  the  tanker  body,  his  motion  sensitive 
peripheral  vision  Is  detecting  "oil  cues  out  toward 
the  wing  tip. 

The  outboard  engine  provides  significant  fore 
and  aft  cues  as  has  already  been  mentioned.  Some 
pilots  develop  a  technique  of  using  the  engine 
nacelle  as  It  Is  bisected  by  the  window  post, 
others  use  the  tall  cone  of  the  engine.  When  the 
receiver  aircraft  Is  centered  In  the  refueling 
envelope,  the  60°h  FOV  provides  a  view  of  most  of 
the  outboard  engine  to  the  pilot.  This  view  of  the 
engine  Is  diminished  as  the  receiver  aircraft  moves 
toward  the  Inner  limits  of  the  envelope.  The  60°h 
FOV  has  provided  an  acceptable,  although  marginal, 
view  of  the  important  outboard  engine  cue. 

Both  the  presently  used  60°h  FOV  and  a  120°h 
FOV  have  been  used  In  the  aerial  refueling 
simulation  at  Boeing.  Very  early  In  the  program  the 
120'h  FOV  was  tested  to  determine  If  adequate  cue 
resolution  could  be  maintained  with  the  wider  field 
of  view  which  Incorporated  the  tanker  wing  tips. 

The  wide  field  of  view  provides  only  about  one-half 
the  resolution  of  the  tanner  features  used  as 
refueling  cues,  and  as  might  be  expected,  this  loss 
In  cue  resolution  was  not  an  acceptable  trade  for 
the  added  roll  cue  afforded  by  being  able  to  see 
the  tanker  wing  tips. 


*<*...  r  •  \1 


435 


The  high  resolution  T.V.  camera,  T.V. 
projector,  and  video  electronics  have  been  replaced 
In  the  simulation  since  the  time  of  the  field  of 
view  test.  An  approximate  25  percent  Increase  In 
displayed  T.V.  resolution  has  been  achieved  with 
this  state-of-the-art  T.V.  and  solid  state  video 
electronics.  This  new  equipment  has  noticeably 
enhanced  the  tanker  Image  quality  at  60*h  FOV, 
however,  while  no  tests  have  been  conducted,  It  Is 
still  not  expected  to  provide  sufficient  resolution 
at  120°h  FOV. 

A  "quick  and  dirty"  evaluation  of  a  color  T.V. 
system  was  accomplished  In  the  spring  of  1981.  The 
color  camera  was  manually  positioned  behind  the 
tanker  model  and  the  static  tanker  Image  was 
projected  on  the  screen  In  front  of  the  cockpit. 
This  visual  Impression  was  extremely  dramatic.  With 
approximately  one  half  the  T.V.  line  resolution  of 
the  existing  monochrome  system,  the  tanker  features 
were  much  more  dlscernable.  Experienced  refueling 
pilots  were  convinced  of  the  significantly  Improved 
Image  quality  and  usefulness  of  the  tanker  cues. 
Based  on  this  short  test,  It  Is  apparent  that  some 
measure  beyond  that  found  In  current  visual  system 
specifications  Is  needed  to  define  Image  quality 
reqlrements.  Unfortunately  for  the  refueling 
simulation,  the  physical  size  of  the  color  T.V. 
camera  precludes  Incorporation  In  the  simulation 
system  without  major  redesign. 

Experience  and  observation  over  the  past  five 
years  suggest  that  a  number  of  Image  quality 
conclusions  for  aerial  refueling  simulation  are 
reasonable.  First,  Image  quality  should  be  defined 
In  terms  of  the  training  cues  that  must  be 
provided.  These  cues  Include  not  only  the  specific 
tanker  features  but  how  much  of  the  tanker  (FOV) 
must  be  Included.  Because  of  the  Interaction 
between  cue  resolution  and  field  of  view,  the 
elements  comprising  Image  quality  should  be 
priori tzed.  Second,  It  Is  readily  apparent  that 
current  50°h  FOVs  provide  acceptable  Image  quality. 
The  potential  for  Inclusion  of  more  tanker 
features/cues  through  futher  expansion  of  the  field 
of  view  appears  possible  with  existing  camera/model 
techniques.  The  field  of  view  certainly  can  Include 
the  entire  outboard  engine  without  resulting  In 
unacceptable  Image  quality;  expansion  of  the  field 
of  view  to  as  much  as  90®h  may  even  be  possible 
without  resorting  to  a  two  channel  visual  system 
with  Its  higher  associated  costs. 

SUMMARY  AND  CONCLUSION 

Five  years  of  evolving  aerial  refueling 
simulation  training  experience  has  provided  a 
relatively  unique  opportunity  to  observe  the 
effects  of  simulator  design  and  fidelity  on 
training  effectiveness.  Just  because  a  simulator 
design  does  not  make  use  of  the  latest  technology. 


or  provide  the  utmost  In  fidelity,  does  not  mean 
that  It  Is  not  an  effective  approach  for  training. 
The  state  of  the  art  of  a  simulator  may  be  measured 
more  by  the  training  provided  than  by  the 
technology  employed. 

For  aerial  refueling  training,  a  fixed  base, 
generic  cockpit  simulator,  with  a  wide  field  of 
view  visual  display,  using  the  "old"  camera/model 
Image  generation  approach  has  been  found  to  provide 
extremely  effective  training  for  a  range  of 
different  aircraft  types.  In  many  areas,  the 
traditional  notion  that  better  fidelity  means 
better  training  has  been  opened  to  question, 
particularly  In  terms  of  the  cockpit  and 
Instruments,  and  perhaps  In  terms  of  handling 
qualities.  The  costly  provisioning  of  a  multitude 
of  Instructional  features  can  be  questioned  and 
cost/effectively  limited  to  what  the  Instructor  can 
practically  use  In  day  to  day  training.  The 
training  gaming  area  should  be  looked  at  In  terms 
of  what  Is  needed  for  effective  training  rather 
that  the  too  often  used  "nice  to  have" .  The 
Implications  for  creative  and  cost/effective  design 
approaches  are  significant  when  the  training  need 
Is  understood.  Even  when  seemingly  contradictory 
requirements  of  Image  quality  and  field  of  view  are 
considered,  the  state-of-the-art  answer  may  well  be 
better  satisfied  by  an  old  design  approach  than  by 
the  latest  technology  achievements. 

What  Is  required  Is  more  and  better  training, 
at  a  lower  cost.  Where  advanced  technology  supports 
this  objective.  It  should  be  used;  where 
application  of  existing  technology  Is  more 
cost/effective,  the  choice  Is  clear.  The  real 
answer  may  be  found  In  understanding  the  training 
requirements,  limiting  the  scope  of  simulator 
design  to  meet  the  training  need,  and  then  applying 
all  available  design  approaches  to  the 
cost/effectiveness  test.  If  this  tailoring  of 
technology  to  the  training  application  results  In 
an  apparent  backward  step  In  technology 
advancement,  perhaps  It  still  can  be  considered 
Innovative  In  terms  of  training  productivity. 
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ABSTRACT 

This  paper  describes  a  useful  management  tool  for  helping  program  managers 
determine  the  cost-effectiveness  of  competing  maintenance,  engineering  support, 
and  other  types  of  service  contracts.  The  underlying  analytical  framework  is 
based  upon  cost  improvement  (learning)  curve  theory  and  incorporates  the  ef¬ 
fects  of  competition  on  this  process.  Supporting  data  are  discussed.  The  ori¬ 
ginal  research  was  performed  by  the  authors  at  The  Analytic  Sciences  Corporation 
(TASC)  for  the  Air  Force  Human  Resource  Laboratory's  (AFHRL)  Advanced  Simulator 
for  Pilot  Training  (ASPT) ,  Williams  AFB,  but  is  applicable  to  other  simulator 
maintenance  and  support  contracts  where  the  work  is  of  a  repetitive,  technical, 
and  somewhat  complex  nature. 


INTRODUCTION 

Numerous  studies  have  examined  the 
impact  of  competition  on  acquisition 
costs .  Several  of  these  analyses  have 
developed  useful  models  and  predicted  ac¬ 
quisition  costs  for  military  hardware 
with  some  highly  sophisticated  efforts  re¬ 
cently  prepared.  Few  research  efforts 
have  analyzed  the  effects  of  competition 
on  service-type  contracts.  Most  studies 
concerning  competition  for  service  con¬ 
tracts  are  qualitative  discussions  with 
little  or  no  attempt  to  develop  an  analy¬ 
tical  framework  or  a  supporting  data  base. 

Questions  which  program  management 
must  often  encounter  and  whose  answers 
benefit  from  the  use  of  a  predictive 
framework  include  the  following: 

•  What  are  likely  future  program 
costs  of  continuing  to  use  a 
sole-source  contractor? 

•  What  are  expected  costs  from 
competing  the  service  contract? 

•  Does  the  accumulated  expe¬ 
rience  and  knowledge  of  the 
original  sole-source  firm  pro¬ 
vide  too  great  an  obstacle 
for  a  potential  competitor  to 
overcome? 

•  Are  new  firms  able  to  effec¬ 
tively  compete  with  the  ori¬ 
ginal  contractor? 

•  Are  potential  savings  from 
competition  large  enough  to 
recover  the  up-front  or  non¬ 


recurring  costs  of  competing 
a  contract  (e.g.,  costs  of 
technology  transfer  to  poten¬ 
tial  competitors)? 

•  Will  contractor  performance 
deteriorate  from  intense  com¬ 
petition  and  affect  success¬ 
ful  attainment  of  program 
goals? 

While  performing  research  for  the 
Air  Force  Business  Research  Management 
Center  (AFBRMC)  and  the  Air  Force  Human 
Resource  Lab  (AFHRL) ,  the  authors  have 
developed  a  framework  for  analyzing  cost- 
performance  trade-offs  and  predicting 
costs  of  service  contracts.  In  addition, 
the  authors  have  incorporated  the  effects 
of  competitioi  into  the  framework  and  have 
developed  a  supporting  data  base  and  com¬ 
puter  model. 

The  authors '  original  research  in 
this  area  was  performed  and  applied  for  an 
analysis  of  flight  simulator  maintenance 
and  support  contracts.  Although  specific 
examples  are  from  the  flight  simulator 
case,  the  concepts  which  are  discussed  ap¬ 
ply  to  service  contracts  in  general. 


DEVELOPING  THE  ANALYTICAL  FRAMEWORK 

In  analyzing  the  underlying  theoreti¬ 
cal  concepts  of  competitive  service  con¬ 
tracting,  the  authors  found  several  con¬ 
cepts  which  were  similar  to  those  incorpo¬ 
rated  into  some  of  the  previous  hardware 
acquisition  studies.  Principal  among 
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these  were  learning  or  cost  improvement 
curves.  Since  some  of  the  services  which 
the  authors  initially  examined  were  highly 
complex  (often  state-of-the-art)  support 
functions,  it  intuitively  made  sense  that 
workers,  supervisors,  and  management  would 
improve  their  efficiency  as  familiarity 
with,  and  knowledge  of,  the  system  grew. 

This  paper  begins  with  a  brief  gen¬ 
eral  discussion  of  cost  improvement 
(learning)  curve  theory  and  then  illus¬ 
trates  important  differences  in  unique 
application  of  the  theory  to  service  con¬ 
tracts  , 

COST  IMPROVEMENT  CURVE  THEORY 

A  cost  improvement  curve  reflects 
the  relationship  between  the  unit  cost 
(or  unit  price)  of  an  item  and  the  quan¬ 
tity  of  the  item  produced.  An  "80  per¬ 
cent"  curve  is  one  in  which  a  doubling  of 
output  drives  unit  cost  down  to  80  percent 
of  its  initial  value.  That  is,  the  cost 
of  the  2Nth  unit  is  20  percent  less  than 
the  cost  of  the  Nth  unit. 

Simply  stated,  the  cost  improvement 
curve  represents  the  decreasing  costs  of 
accomplishing  any  repetitive  operation  as 
the  operation  is  continued.  As  any  tech¬ 
nically  complex  process  is  repeated, 
whether  it  is  production  of  missiles  or 
maintenance  of  simulators,  employee  and 
supervisory  familiarity  should  grow,  tech¬ 
nical  methods  should  improve,  and  manager¬ 
ial  efficiency  should  increase. 

Cost  Improvement  Curve  in  Standard  Form 

Graphically  displayed  in  Figure  1  is 
a  cost  improvement  curve  in  standard  fora. 
The  curve  shows  the  recurring  cost  of  each 
unit  of  production  as  a  function  of  total 
quantity  produced.  The  area  under  the 
curve  is  the  total  recurring  cost  of  a 
given  production  run. 


Quantity 


It  is  important  to  note  the  rapid 
cost  improvement  which  is  incurred  early 
in  the  performance  period.  Each  horizon¬ 
tal  increment  of  progress  yields  a  smaller 
absolute  reduction  in  cost  than  the  pre¬ 
vious  increment .  As  the  curve  grows  in¬ 
creasingly  flatter,  larger  amounts  of  pro¬ 
duction  or  work  are  required  in  order  to 
further  reduce  cost  until  the  cost  im¬ 
provement  curve  ceases  to  function  as  a 
useful  tool.  Producers  and  suppliers  of 
relatively  limited  production  quantities , 
or  state-of-the-art  engineering  services 
(e.g.,  simulator  maintenance),  are  rarely 
found  on  the  flat  end  of  the  curve. 

SERVICE  COST  IMPROVEMENT  CURVES 

Cost  improvement  analysis  has  been 
applied  almost  exclusively  to  production 
studies  (e.g.,  missiles,  aircraft,  new 
consumer  products)  with  little  application 
to  services.  The  theory  fits  service  con¬ 
tract  analysis,  but  requires  modifications 
from  production  or  hardware  application. 

Not  all  services,  just  as  not  all 
production  situations,  are  suitable  for 
cost  improvement  durve  analysis.  As  for 
production  studies,  the  cost  improvement 
curve  effects  for  services  are  most 
notable  for  relatively  new  or  complex 
situations.  For  example,  the  cost  im¬ 
provement  curve  for  custodial  services, 
after  a  small  and  rapid  period  of  cost  im¬ 
provement  ,  is  probably  extremely  flat . 

Most  janitorial  companies,  or  individuals, 
are  well  established  on  the  flat  part  of 
the  cost  improvement  curve .  But  for  ser¬ 
vices  where  techniques  used,  skills  re¬ 
quired,  and  equipment  serviced,  are  rela¬ 
tively  new,  complex,  changing  or  diverse 
in  nature,  cost  Improvement  analysis  pro¬ 
vides  a  useful  analytical  tool. 

The  authors  have  identified  several 
major  features  of  cost  improvement  curve 
theory  which  are  applicable  to  service 
contracts.  A  discussion  of  those  features 
follows . 

Definition  of  an  Increment  of  Progress 

One  immediate  need  for  applying  this 
type  of  analysis  to  service  contracts  is 
definition  of  an  increment  of  progress 
(i.e.,  a  unit  of  production)  along  the 
cost  improvement  curve.  Some  period  of 
performance  is  obviously  the  most  conven¬ 
ient  way  to  measure  quantity  (see  Figure 
2) .  The  unit  period  of  performance  chosen 
should  best  reflect  the  repetitive  nature 
of  the  work  and  is  determined  analytically. 
For  flight  simulator  maintenance,  the 
choice  of  a  suitable  increment  of  progress 
is  not  easily  identifiable. 


Figure  1  Cost  Improvement  Curve 
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Figure  2  Service  Cost  Improvement 
Curves 


Although  set  daily  routines  exist,  (train¬ 
ing,  repairs,  data  analysis,  preventive 
maintenance,  etc.),  the  same  problems  do 
not  arise  each  day,  indicating  that  the 
best  incremental  measure  of  progress  is 
longer  than  a  day.  With  a  period  of  per¬ 
formance  measured  along  the  horizontal 
axis,  the  vertical  axis  will  now  reflect 
the  cost  of  service  for  each  successive 
period  (i.e.,  incremental  cost). 

For  example,  to  define  a  suitable 
increment  of  progress  for  a  flight  simu¬ 
lator  case,  the  authors  have  calculated 
actual  cost  improvement  curve  rates  using 
various  increments  of  progress.  The  re¬ 
sults  are  presented  in  Table  1  and  are 
based  on  contract  data  obtained  from  the 
Simulator  for  Air-to-Air  Combat  (SAAC) 
Program  Office  at  Luke  AFB.  As  the  incre¬ 
ments  become  smaller,  the  effect  which 
they  have  upon  the  cost  improvement  rate 
becomes  less  significant.  The  cost  im¬ 
provement  rate  varies  little  with  incre¬ 
ments  of  one  month  or  less.  Consequently, 
the  authors  selected  one  month  as  the  in¬ 
crement  of  progress  for  analysis  purposes. 


Table  1 

SAAC  Cost  Improvement  Curve  Rates 


Increment  Cost  Improvement 

of  Progress  Curve  Rate  _ 


Fiscal  Year  ,84 

Month  .895 

Week  .902 

Day  .904 

Hour  .905 


Table  2 

Cost  Improvement  Curve  Rates  for  Flight 
Simulator  Maintenance  and  Support 


System 

Rate 

F-4E/A-7D  AAFTS 

86.2 

C-5/C-141 

91.8 

F-15 

90.8 

E-3A 

95.3 

F-111A-F 

86.5 

SAAC 

89.5 

ASPT 

90.0 

Average 

90.0 

Acceptable  increments  of  progress  can 
be  determined  for  other  types  of  service 
contracts  and  will  vary  from  case  to  case. 

Level  of  Effort,  the  Cost  Improvement 
Forces,  and  Contractor  Performance 

For  many  service  contracts,  a  speci¬ 
fied  level  of  effort  (manpower)  is  re¬ 
quired.  In  these  cases,  the  contractor  is 
required  to  provide  a  total  number  of  man¬ 
hours  of  service  during  the  contract 
period.  The  contractor  may  be  required  to 
provide  a  fixed  daily  level  of  manpower  or 
he  may  only  have  the total  manpower  level 
fixed,  remaining  free  to  choose  his  alloca¬ 
tion  (e.g.,  resources  during  a  major  system 
failure) .  Cost  improvement  effects  are  not 
eliminated  by  this  requirement,  but  simply 
emerge  in  a  different  manner  than  the  usual 
cost  improvement  process. 

With  daily  (or  monthly)  manpower 
levels  fixed,  employees  will  learn,  tech¬ 
nical  skills  will  develop,  and  managerial 
efficiency  will  increase.  These  forces 
produce  an  improvement  in  contractor  per¬ 
formance  over  the  length  of  the  contract. 

If  manpower  and  skills  are  allocated  to 
meet  required  performance  standards  at  the 
beginning  of  the  contract  period  and  remain 
fixed  throughout  the  contract,  then  the 
contractor's  performance  levels  will  in¬ 
creasingly  exceed  the  required  standards  as 
time  passes  and  as  contractor  experience 
grows .  The  improved  contractor  performance 
represents  the  learning  process  within  a 
contract  period.*  This  effort  has  been 
observed  for  several  level  of  effort  con¬ 
tracts  , 

During  a  contract  period,  the  govern¬ 
ment  will  be  unable  to  benefit  from  the 
cost  improvement  process  in  terms  of 
reduced  cost,  but  will  receive  an  increase 
in  performance  beyond  that  required.  The 
emergence  of  performance  levels  which 


Using  a  month  as  an  acceptable  incre¬ 
ment  of  progress,  the  authors  have  calcu¬ 
lated  cost  improvement  rates  (Table  2)  for 
a  number  of  flight  simulator  maintenance 
and  support  contracts.  The  average  rate 
is  90  percent. 


*This  analysis  assumes  that  employee  idle 
time  does  not  increase.  If  the  increased 
efficiency  and  learning  emerge  as  en  in¬ 
crease  in  idle  time,  then  performance 
levels  will  remain  relatively  stable. 
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exceed  Chose  required  by  Che  governmenC 
indicates  Chac  Che  governmenC  can  reduce 
specified  manpower  and  skill  levels  and 
realize  a  cose  reduction  for  the  next  con- 
CracC.  This  process  Is  graphically  rep- 
resenced  in  Che  sCairsCep  manner  illus- 
CraCed  In  Figure  3. 


Cost 


Perform¬ 

ance 


Contract 

Period 


Contract 

Period 


Figure  3  Fixed  Level  of  Effort 

Cose  improvement  forces  for  level  of 
effort  contracts  might  emerge  in  a  differ¬ 
ent  manner  than  the  performance  improve¬ 
ment  case  described  above.  Assume  a  con¬ 
tractor  is  obligated  to  provide  a  specified 
number  of  manhours  of  service  for  the  con¬ 
tract  period,  but  is  not  required  to  allo¬ 
cate  those  manhours  equally  over  the  length 
of  the  contract.  He  may  allocate  his  con¬ 
tractually  required  manpower  and  resources 
in  a  manner  that  reflects  the  actual  cost 
improvement  process.  In  this  situation, 
relatively  larger  amounts  of  resources  are 
used  in  providing  service  at  the  beginning 
of  the  contract  period  than  are  used  in 
providing  service  near  the  end  of  the  con¬ 
tract,  If  the  specified  level  of  effort 
for  a  contract  period  is  set  too  high  and 
the  contractor  allocates  his  manpower  and 
costs  by  following  the  actual  cost  im¬ 
provement  curve,  then  performance  will 
remain  stable,  but  at  a  rate  in  excess  of 
the  contractually  required  level .  The 
level  of  stairstep  cost  reduction  possible 
between  contracts  is  still  driven  by  the 
underlying  cost  improvement  forces.  Figure 
4  illustrates  this  situation. 


Cost 


Actual  Resource 
Allocation 

Proposed  Resource 
Allocation 


Contract 

Period 


The  solid  line  (steps)  represents  the 
cost  and  manpower  level  which  was  nego¬ 
tiated  with  the  contractor.  In  reality, 
the  contractor  may  allocate  manpower  and 
skill  levels  according  to  the  descending 
pattern  of  the  dotted  line. 

In  the  health  care  field,  one  of  the 
first  service  areas  where  the  authors ' 
research  revealed  cost- improvement  curve 
forces  at  work,  the  performance-cost 
trade-off  is  easily  identifiable.  Analy¬ 
sis  of  two  Medicaid  claims  processing  con¬ 
tracts,*  one  sole  source  and  one  competi¬ 
tive,  illustrates  the  trade-off.  The  cost 
per  unit  (claim  processed)  of  the  sole 
source  contract  showed  no  cost  improve¬ 
ment  present.  However,  contractor  per¬ 
formance  (error  rate)  continually  improved 
until  it  reached  an  eventual  level  far  in 
excess  of  contract  requirements .  In  the 
case  of  the  competitively  bid  contract, 
cost  per  unit  followed  a  cost  improvement 
curve  and  steadily  declined  over  time. 
Performance  levels  in  this  case  stayed 
essentially  constant.  (The  effects  of 
competition  for  service  contracts  are 
examined  in  more  detail  later  in  this 
paper. ) 

Two  performance-cost  situations  for 
level  of  effort  contracts  have  been  dis¬ 
cussed.  The  first,  where  the  level  of 
manpower  resources  is  held  constant  during 
a  contract  period,  results  in  increasing 
performance  levels.  The  second,  where 
manpov/er  follows  a  decreasing  pattern  with 
time,  provides  relatively  stable  perform¬ 
ance  levels  at  less  cost.  A  combination 
of  these  two  effects  may  also  occur,  A 
contractor  may  allocate  his  manpower  in  a 
decreasing  manner  which  does  not  entirely 
reflect  all  of  the  cost  improvement  for¬ 
ces  .  Performance  levels  will  then  in¬ 
crease,  but  at  a  slower  pace  than  if 
resources  were  equally  distributed  over 
the  entire  contract  period.  Figure  5 
illustrates  the  combination  effect. 
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Figure  5  Combined  Effect 
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*Preliminary  Performance  Profiles  of  Con¬ 
tractors  Operating  Under  Experimental 
Contracts,  Health  Care  Financing  Admin¬ 
istration. 


Figure  4  Decreasing  Level  of  Effort 
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Th«  observation  of  performance-cost 
trade-offs  on  cost  improvement  curves  ia 
unique  to  service  contract  analysis.  In 
the  production  of  hardware,  a  similar  rela¬ 
tionship  may  exist,  but  may  require  years 
before  the  quality  or  performance  trade¬ 
off  is  known.  With  many  service  type  con¬ 
tracts,  the  performance  level  is  usually 
quickly  and  easily  measurable  and  can 
serve  as  a  useful  tool  in  acquisition 
planning  for  follow-on  contracts  or  for 
changing  current  contracts. 

SUMMARY:  ANALYTICAL  FRAMEWORK 

Cost  improvement  curve  theory,  al¬ 
though  emerging  in  a  somewhat  different 
manner  than  in  previous  applications, 
proves  applicable  and  useful  for  perform¬ 
ing  service  contract  cost  analysis.  Few 
government  officials  actively  use  the  cost 
improvement  process  to  project  future  costs 
when  negotiating  service  contracts .  When 
contracting  officers  recognize  a  signifi¬ 
cant  improvement  in  contractor  performance 
during  the  most  recent  contract  period, 
they  will  negotiate  to  reduce  costs  for  the 
next  contract.  This  type  of  passive  use  of 
the  cost  improvement  process  delays  any 
possible  cost  reduction  by  at  least  one 
contract  period,  A  more  active  use  of  this 
type  of  analysis  includes  projecting  costs, 
performance ,  and  requirements  with  the  use 
of  cost  improvement  theory.  Reduced  costs 
and  improved  performance  are  reflected  in 
the  contract  and  captured  as  they  occur. 

An  active  use  of  the  cost  improvement 
process  projects  future  costs  and  assesses 
contracting  options  based  on  these  projec¬ 
tions  . 

EFFECTS  OF  COMPETITION 

Attention  is  now  focused  on  the 
effects  of  competition  on  the  cost  im¬ 
provement  process.  The  benefits  of  com¬ 
petition,  the  relative  cost  positions  of 
potential  competitors ,  and  several  unique 
aspects  of  service  contract  competition 
are  discussed. 

BENEFITS  OF  COMPETITION 

The  authors  have  identified  two  dis¬ 
tinct  effects  on  the  cost  improvement  curve 
due  to  the  introduction  of  competition: 

•  A  downward  shift  of  the  cost 
improvement  curve  (i.e.,  a 
reduction  in  costs  and  profits) 

•  A  downward  rotation  of  the  cost 
improvement  curve  (i.e.,  a 
faster  rate  of  improvement) . 

These  two  effects  are  graphically 
displayed  in  Figure  6.  (In  Figure  6,  the 
standard  cost  improvement  curve  has  been 
transformed  into  a  logarithmic  version. 

This  transformation  produces  a  linear  rela¬ 
tionship  which  is  convenient  for  display¬ 
ing  and  describing  the  analytical  results,) 


It  is  assumed  that  contracts  were  initial¬ 
ly  sole  source.  The  curve's  parallel 
downward  shift  from  A  to  C  results  from 
both  a  reduction  in  profit  and  a  reduction 
in  costs . 


Log  of 
Increment¬ 
al 
Cost 


Lor  of  Time 
Increment 


Figure  6  Benefits  of  Competition 


The  final  reduction  from  C  to  P2  repre¬ 
sents  a  reduction  based  upon  the  firm 
developing,  under  competition,  a  steeper 
cost  improvement  curve  (i.e.,  a  faster 
rate  of  cost  improvement) .  The  line  DP2 
reflects  the  steeper  slope.  1 

Without  the  pressures  of  competition, 
the  contractor  would  have  continued  along 
his  projected  sole  source  cost  improvement 
curve,  Pi  to  A.  The  total  area  in 
P1AQ2Q1  represents  what  the  total  costs 
would  nave  been  if  the  government  had 
remained  with  a  sole  source  producer.  The 
area  DP2Q2QI  represents  the  actual  costs 
obtained  under  competition.  The  area 
PlAP2D  represents  the  amount  of  potential 
savings  due  to  competition. 

It  is  important  to  recognize  that 
training  or  skill  substitution  can  produce 
an  improved  cost  improvement  rate  repre¬ 
sented  by  the  area  DCP2 .  If  such  cost 
improvement  changes  are  ignored,  the  en¬ 
tire  change  AP2  is  likely  to  be  antici¬ 
pated  for  all  future  buys.  However,  it  is 
clear  from  Figure  6  that  the  size  of  CP2 
critically  depends  upon  the  total  number 
of  periods  for  which  service  is  provided. 
If  the  combined  profit  and  cost  savings 
due  to  competition  (the  downward  shift  of 
the  cost  improvement  curve)  can  be  estab¬ 
lished  at  a  given  level,  these  savings  can 
be  ret  jd  for  all  future  periods  of  ser¬ 
vice.  However,  the  savings  from  the  down¬ 
ward  rotation  of  the  cost  improvement 
curve  (at  point  D)  increase  as  the  number 
or  periods  of  service  increase. 

The  potential  advantage  to  the  govern¬ 
ment  from  competing  a  previously  sole 
source  procured  service  was  illustrated  by 
Figure  6.  In  that  figure,  the  cost  to  the 
government  of  purchasing  the  service  is 
easily  identifiable  and  represented  by  the 
area  under  the  cost  improvement  curve. 

Even  if  the  new  competitor  wins  the  com¬ 
petition,  the  cost  benefits  o~i^competition 
to  the  government  are  still  present  and 
represented  by  the  shift  and  rotation  of 


441 


the  coat  improvement  curve.  Regardless  of 
which  competitor  wins,  the  benefits  of 
competition  to  the  government  emerge  in 
the  same  fashion.  Not  only  does  an  ini¬ 
tial  cost  and  profit  reduction  apply  to 
all  future  periods  of  service  affected  by 
the  competition,  but  the  cost  improvement 
process  occurs  at  an  improved  rate.  Thus, 
there  are  both  immediate  and  lasting  eff¬ 
ects  of  competition. 

UNDERLYING  COST  TO  THE  SECOND 
FIRM 

When  faced  with  a  competitive  situa¬ 
tion  for  a  service  previously  procured 
from  a  sole  source  firm,  the  second  firm 
facis  an  underlying  cost  schedule  as 
depicted  in  Figure  7 . 


Figure  7  Underlying  Cost  to  the 
Second  Firm 


Competition  is  conducted  at  time 
period  T.  Because  of  its  lack  of  experi¬ 
ence,  the  second  firm  must  plan  on  begin¬ 
ning  service  for  an  increment  of  time ,  at 
some  price  B,  which  is  in  excess  of  the 
prevailing  sole  source  price  C.  If  the 
second  firm's  cost  improvement  rate  is 
steeper  than  that  for  the  sole  source 
firm,  cost  parity  will  be  achieved  at 
some  point  D.  If,  after  reaching  Point  D, 
the  second  firm  were  to  move  down  the  pro¬ 
jected  single  source  curve  to  point  F,  his 
total  cost  would  exceed  the  projected 
single  source  total  cost  by  the  amount 
BCD.  This  is  the  experience  cost  for  the 
second  firm.  If  this  was  the  basis  of  the 
second  firm's  bid,  the  original  sole 
source  firm  would  then  win  the  competition 
by  simply  proposing  a  cost  reflecting  a 
move  from  point  C  to  point  F  on  its  cost 
improvement  curve.  He  would  not  feel  any 
competitive  pressure  and  would  not  have  to 
reduce  costs,  profits,  or  improve  his  cost 
improvement  curve  rate  to  stay  competitive 
(i.e.,  he  would  not  have  to  shift  and 
rotate  his  cost  improvement  curve) . 

The  second  firm,  in  an  attempt  to  win 
the  competition,  must  continue  down  a 
curve,  after  reaching  point  D,  which  is 
steeper  than  the  projected  sole  source 
curve,  such  as  from  D  to  G.  The  total 
amount  of  potential  competitive  savings 
over  the  projected  single  source  cost  in 


this  case  is  equal  to  the  area  DFG  minus 
the  area  BCD  for  the  contract  period. 

When  area  DFG  is  greater  than  area  BCD, 
the  second  firm  is  able  to  overcome  his 
"experience"  cost  and  offer  service  to  the 
government  at  a  lower  cost  than  what  would 
have  prevailed  from  continuation  of  the 
sole  source  situation. 

The  sole  source  firm,  in  order  to  win 
the  competition,  must  produce  a  lower  cost 
than  what  he  estimates  his  competitor  will 
bid.  This  lowering  of  cost  by  the  sole 
source  firm  is  evidenced  by  a  shift  and 
rotation  (as  earlier  discussed)  in  the 
sole  source' cost  improvement  curve.  If 
the  sole  source  firm  miscalculates  and  his 
bid  does  not  reflect  a  large  enough  shift 
and  rotation  to  overtake  his  competitor's 
proposed  cost,  the  second  firm  will  win 
the  competition.  In  either  event,  the 
government  benefits  from  the  competition. 

Several  factors  will  help  the  second 
firm  obtain  a  competitive  cost  position. 
These  factors  include  the  following: 

e  Immediate  competitive  pressure 

•  Contract  length 

a  Performance-cost  trade-offs 

•  System  modifications. 

Immediate  Competitive  Pressure 

In  contrast  to  the  sole  source  firm, 
the  second  firm  would  begin  providing 
service  at  costs  which  were  competitively 
bid  and  responsive  to  pressures  of  com¬ 
petition.  The  shift  and  rotation  of  his 
underlying  cost  schedule  are  reflected  for 
his  first  period  of  service.  Figure  8 
below,  which  is  similar  but  not  identical 


Figure  8  Immediate  Competitive 

Pressure  on  the  Second  Firm 


to  Figure  7,  helps  to  illustrate  this 
point.  Figure  7  is  drawn  with  point  B, 
the  cost  of  the  second  firm's  initial 
period  of  service,  lower  than  point  A,  the 
cost  of  the  sole  source  contractor's  ini¬ 
tial  period  of  service.  Since  the  new 
firm  is  immediately  faced  with  a  competi¬ 
tive  situation,  he  will  logically  begin, 
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or  bid  to  begin,  providing  service  at  a 
lower  cost  level  and  with  a  more  rapid 
improvement  rate  than  that  which  pre¬ 
vailed  at  the  sole  source  determined 
point  A.  This  reduction  in  the  initial 
cost  is  caused  by  factors  such  as  reduc¬ 
tion  in  profit,  technology  transfer,  and 
skill  substitution.  Additionally,  the 
second  firm's  cost  improvement  curve  is 
more  steeply  sloped  than  the  sole  source 
curve,  with  the  intersection  of  the  curves 
(cost  parity)  occuring  at  point  D.  Fac¬ 
tors  such  as  improved  processes  or  more 
experienced  employees  will  accelerate  the 
cost  improvement  rate. 

If  point  B  is  placed  at  a  cost  level 
reflecting  a  10%  downward  shift  from  point 
A,  and  if  a  4  percentage  point  rotation  is 
used  for  the  second  firm's  cost  improve¬ 
ment  rate  (values  which  are  well  within 
the  range  of  observed  data  cases) ,  then 
the  period  of  time  from  T2  to  T3  (the  time 
required  for  the  second  firm  to  move  from 
point  B  to  point  D  and  achieve  cost  pari¬ 
ty)  is  equal  to  approximately  5%  of  the 
period  of  time  from  to  T3  (the  time 
required  for  the  sole  source  firm  to  move 
from  point  A  to  point  D)  .  Simply  stated, 
with  the  above  assumptions ,  the  second 
firm  requires  only  5 %  of  the  service  ex¬ 
perience  that  the  sole  source  contractor 
needed  (assuming  a  90%,  sole  source  cost 
improvement  curve)  in  arriving  at  the 
cost  level  represented  by  point  D.  For 
example,  if  the  sole  source  firm  has  five 
years  of  sole  source  experience,  the 
second  firm  would  need  less  than  four 
months  to  obtain  cost  parity. 

Although  these  numbers  are  only  an 
example,  they  help  illustrate  the  magni¬ 
tude  of  the  competitive  pressures  on  the 
original  sole  source  contractor.  When 
shift  and  rotation  parameters  consistent 
with  observed  data  on  competitive  pro¬ 
curements  are  applied  to  a  new  competi¬ 
tor's  underlying  cost  improvement  curve 
at  the  initial  period  of  service,  the 
original  contractor's  cost  advantage,  pro¬ 
jected  by  the  sole  source  curve,  Is  rapid¬ 
ly  overcome . 

It  is  important  to  note,  that  unlike 
a  microeconomic  analysis,  inefficiency 
and  waste  are  assumed  to  exist  for  the 
sole  source  situation.  In  a  microeco¬ 
nomic  analysis,  when  faced  with  the  pres¬ 
sures  of  the  market,  the  firm  operates 
with  the  most  efficient  combination  of 
resources  possible.  This  situation  is 
not  true  for  some  federal  contracts , 
where  the  market  forces  are  either  absent 
or  hidden  under  a  totally  unique  buyer- 
seller  relationship.  The  sole  source 
firm,  although  increasing  his  efficiency 
as  his  experience  grows  (i.e.,  follows  a 
cost  improvement  curve) ,  is  not  following 
the  most  efficient  (optimal)  cost  improve- 
ment  path .  The  pressures  of  competition 
are  required  to  move  the  sole  source  firm 
toward  the  optimal  curve. 


When  a  new  competitor  bids  on  the  con¬ 
tract,  since  he  is  immediately  faced  with 
a  competitive  situation,  his  bid  for  the 
initial  cost  of  work  will  not  reflect  the 
initial  sole  source  cost  but  will  be  based 
upon  an  improved  level  of  efficiency. 

Even  if  the  new  competitor  bid  to  operate 
on  his  optimal  curve,  the  sole  source  firm 
would  still  retain  some  amount  of  cost 
advantage  (assuming  the  sole  source  firm 
bid  to  operate  on  or  near  his  optimal 
curve)  because  of  his  initial  sole  source 
experience.  In  industries  where  new  tech¬ 
nical  knowledge  disseminates  relatively 
rapidly  among  firms,  this  advantage  may 
prove  insignificant.  In  this  case,  the 
new  firm  and  the  sole  source  firm  would 
face  approximately  equal  "optimal"  cost 
improvement  paths.  For  example,  in  the 
electronics  industry,  when  one  firm 
achieves  a  technological  breakthrough,  the 
new  knowledge  spreads  rapidly  throughout 
the  industry.  As  a  sole  source  firm  gains 
its  initial  experience,  word  of  the  tech¬ 
niques  developed  and  knowledge  acquired 
easily  spreads  to  other  firms. 

For  a  case  where  the  sole  source  firm 
has  closely  guarded  technological  and  pro¬ 
cedural  developments,  its  advantage  is 
much  more  significant.  In  this  latter 
case,  the  initial  sole  source  firm  should 
win  the  competition  by  moving  close  enough 
to  its  optimal  cost  improvement  curve  so 
that  its  "learning"  or  "experience."  advan¬ 
tage  cannot  be  overcome  by  a  new  and  inex¬ 
perienced  competitor.  If  both  firms  bid 
to  provide  service  at  their  optimal  levels, 
the  sole  source  firm  will  win,  providing 
it  has  an  initial  advantage  from  prior 
experience  with  the  system.  Of  coarse, 
since  the  sole  source  firm  improves  its 
efficiency,  the  government  will  realize  a 
savings'!  If  the  sole  source  firm  mis- 
judges  its  competition  and  bids  too  high, 
the  more  efficient  new  firm  will  win  the 
competition  and  still  produce  the  savings 
to  the  government. 

Whether  or  not  the  experience  and 
knowledge  of  the  sole  source  firm  provides 
a  significant  obstacle  for  a  new  competi¬ 
tor  is  dependent  on  the  nature  of  the  in¬ 
dustry,  the  type  of  service  provided,  and 
the  availability  of  technical  data.  '"Im¬ 
plicit  in  this  analysis  is  that  rights  to 
unique  technical  and  procedural  data  are 
invaluable  items  to  write  into  a  con¬ 
tract.)  Careful  analysis  of  these  factors 
is  required  to  determine  and  quantify  the 
likely  cost  improvement  curve  of  the  new 
firm. 

Contract  Length 

Contract  length  is  crucial  to  a  poten¬ 
tial  second  source  in  many  service  con¬ 
tract  situations.  With  a  lengthy  contract 
horizon  and  an  accelerated  rate  of  cost 
improvement,  a  second  firm  not  only  has 
ample  time  to  achieve  cost  parity,  but 
also  acquires  sufficient  cost  reduction 
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(learning)  to  overcome  the  experience 
cost  of  achieving  parity  and  produce  a 
substantial  savings  to  the  government. 

During  a  recent  competition  for  a 
flight  simulator  maintenance  and  support 
contract,  the  second  firm  won  the  award 
over  the  original  sole  source  firm  for  a 
five-year  period.  The  second  firm's  costs 
were  based  on  providing  support  at  the 
same  average  monthly  cost  (constant  year 
dollars)  for  the  entire  five  years.  With 
a  firm- fixed  price  (FFP)  contract  and  per¬ 
formance  rather  than  level  of  effort  re¬ 
quirements,  the  winning  firm  will  likely 
follow  a  cost  improvement  curve  as  depic¬ 
ted  in  Figure  9.  As  the  firm's  experience 
grows,  the  cost  improvement  forces  drive 
the  monthly  cost  of  service  below  the 
average  monthly  cost  for  the  five-year 
period  of  performance. 


Figure  9  Actual  Versus  Monthly 
Average  Cost 

The  actual  cost  curve  of  the  winning 
firm  must  also  intersect  the  projected 
sole  source  cost,  improvement  curve  (a3 
seen  earlier  in  Figure  7) .  With  a  five- 
year  service  horizon  and  an  accelerated 
rate  of  cost  improvement,  the  winning  firm 
had  ample  time  not  only  .:o  acKievfe  cost 
parity  but  also  to  overcome  the  experi¬ 
ence  cost  of  achieving  parity  and  produce 
a  subs'tanHairiAQ'^)'  sayings  to  the  govern¬ 
ment  over  projected  sole  source  costs. 

Cos t-Per f ormance  Trade-Offs 


As  discussed  earlier,  when  cortract 
performance  standards  are  steadily  ex¬ 
ceeded  by  substantial  amounts,  cost  sav¬ 
ings  are  made  possible  by  employing  less 
resources  (or  less  costly  resources)  and 
lowering  unnecessarily  high  performance 
levels.  This  situation  often  exists  for 
flight  simulator  support.  The  contrac¬ 
tor's  performance  levals  often  exceed  the 
minimum  standards  set  bv  the  government. 

A  new  competitor  can  bid  to  provide  sup¬ 
port  service  at  a  reduced  cost  with  lovrer 
resultant  performance  levels,  but  still 
meet  required  performance  standards. 

For  example,  on  one  USAF  simulator, 
the  sole  source  contractor  has  steadily 
maintained  an  availability  rate  substan¬ 
tially  in  excess  of  the  required  90%.  On 


recent  monthly  performance  reports,  the 
average  availability  rate  approximates 
97%,  with  several  months  displaying  a  99% 
rate.  This  situation  indicates  that  a  new 
competitor  has  sufficient  room  to  reduce 
costs  and  ~s  till  provide  service  at  a  level 
which  meets  performance  requirements .  TF" 
we  assume  that  the  relationship  between 
cost  and  availability  rate  is  similar  to 
Figure  10,  then  the  government  is  spending 
a  disproportionately  large  amount  of  funds 
for  the  excess  performance. 

The  curve  depicted  in  Figure  10 
resembles  a  total  product  curve  for  labor. 
Since  labor  is  the  primary  input  for  a 
support  service  contract,  this  comparison 
appears  both  appropriate  and  useful .  The 
slope  of  the  curve,  which  represents  the 
marginal  product  of  labor,  reflects  the 
diminishing  returns  to  scale  of  labor  as 
additional  amounts  of  that  input  are  sup¬ 
plied.  Additional  gains  in  availability 
become  relatively  more  expensive  in  terms 
of  manpower. 
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Figure  10  Potential  Savings  From  a 

Reduction  in  Availability  Rate 

This  relationship  indicates  that  sub¬ 
stantial  savings  are  possible  by  providing 
service  at  a  slightly  lower  availability 
rate .  A  potential  competitor  may  lessen 
the  sole  source  firm's  cost  advantage  by 
submitting  a  bid  which  is  based  on  pro¬ 
viding  service  at  a  90  percent  availabili¬ 
ty  rate.  The  difference  (in  cost)  between 
the  sole  source  position  on  his  cost  im¬ 
provement  curve  and  the  level  at  which  a 
new  competitor  will  begin  service  is  sub¬ 
stantially  reduced  if  the  sole  source 
firm  bids  on  the  basis  of  a  98%  avail¬ 
ability  level  and  the  new  competitor  bids 
on  the  basis  of  the  required  90%  level. 

As  Figure  10  shows,  a  slight  reduction  in 
availability  produces  a  more  than  propor¬ 
tionate  reduction  in  cost.  Any  move  by 
the  sole  source  firm  to  adjust  his  bid 
downward  on  the  basis  of  a  lower  avail¬ 
ability  rate  shifts  his  costs  below  the 
projected  sole  source  costs  and  is  a 
direct  savings  to  the  government. 
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Figure  11  Competition  and  the 
Availability  Rate 

One  theory  states  that  competition 
produces  improved  performance  at  a  reduced 
cost.  As  figure  11  shows,  this  theory  is 
not  inconsistent  with  the  performance -cost 
relationship  theory.  An  increase  in  effi¬ 
ciency  (e.g.,  improved  processes,  more 
qualified  supervisors,  mere  skilled  em¬ 
ployees)  will  shift  the  marginal  curve  up¬ 
ward.  It  now  becomes  possible  to  sustain 
a  given  level  of  performance  (availability 
rate)  at  less  cost  than  on  the  original 
curve .  The  performance-cost  relationship 
is  still  applicable,  but  occurs  at  an 
improved  level  of  efficiency. 

System  Modifications 


After  each  shift  upward  due  to  modifi¬ 
cations,  the  cost  improvement  process  con¬ 
tinues,  but  a  higher  level  of  costs.  For 
small  changes  (i,e.,  relatively  minor  sys¬ 
tem  modifications) ,  the  upward  shift  is 
negligible  and  will  represent  only  a  small 
portion  of  the  total  cost.  Major  system 
modifications  which  drastically  alter  the 
nature  of  the  support  service  are  graphi¬ 
cally  displayed  as  large  upward  shifts  of 
the  cost  improvement  curve  and  virtually 
represent  a  new  curve  with  significant  cost 
reductions  during  the  first  few  periods  of 
service. 

To  understand  how  this  process  helps 
bring  the  second  firm  into  cost  parity  with 
the  single  source,  the  cost  improvement 
curve  is  viewed  as  a  combination  of  smaller 
cost  improvement  curves.  Theoretically , 
the  cost  improvement  curves  for  each  modi¬ 
fication  and  the  cost  improvement  curves 
for  these  parts  of  the  original  system 
which  are  still  intact  will  sum  to  the 
total  support  cost  for  the  entire  system. 

If  the  modifications  are  recent,  the  sole 
source  firm  has  not  moved  very  far  down  the 
modification  cost  improvement  curve  and  his 
cost  advantage  is  reduced. 

Although  the  effect  of  system  modifi¬ 
cations  is  difficult  to  measure  based  on 
available  data,  it  is  important  to  consider 
its  possible  impact  when  examining  the 
original  cost  disadvantage  of  the  second 
firm. 


Finally,  modifications  to  the  system 
for  which  the  sole  source  contractor  is 
providing  support  help  to  enhance  the  new 
firm's  competitive  position  by  partially 
under-cutting  the  sole  source  cost  advan¬ 
tage. 

Cost  improvement  curve  theory  implies 
continual  improvement  at  repetitious  work. 
If  the  nature  of  the  work  were  to  change. 
then  the  shape  of  the  curve  would  also 
changed  It  the  system  being  maintained 
on  an  engineering  .services  contract  were 
modified,  then  the  nature  of  the  work  will 
change .  Additional  maintenance  checks  are 
required,  additional  workers  are  often 
hired,  spares  are  replaced.  The  effect 
of  this  type  of  change  is  illustrated  by 
the.  modified  cost  improvement  curve  in 
Figure  12. 


Figure  12  Effects  of  Modification  and 
Changes  in  the  Scope  of  Work 


SUMMARY:  EFFECTS  OF  COMPETITION 

The  authors'  research  and  analysis  of 
competition  has  produced  the  following 
results : 

•  Competition  produces  two  dis¬ 
tinct  effects  on  the  cost 
improvement  curve 

A  downward  shift  of  the  curve 

—  A  downward  rotation  of  the 
curve 

The  magnitude  of  these  effects 
determine  the  amount  of  the  poten¬ 
tial  savings  due  to  competition 

•  A  new  competitor  is  faced  with 
an  underlying  cost  disadvantage 
because  of  the  experience  of  the 
sole  source  firm.  The  new  com¬ 
petitor  must  rapidly  overcome 
this  disadvantage,  or  appear  to 
the  original  source  that  he  can, 
if  the  government  is  to  realize 
substantial  savings  from  compe¬ 
tition 

•  Several  factors  exist  which  help 
the  second  firm  in  obtaining  a 
competitive  cost  position 
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Immediate  competitive  pressure 
on  the  second  firm  reduces 
his  initial  cost  of  provid¬ 
ing  service  and  hastens  his 
rate  of  cost  improvement  over 
what  would  prevail  in  a  sole 
source  situation 

—  Sufficient  contract  length 

provides  a  potential  new  firm, 
who  follows  a  more  rapid  rate 
of  cost  reduction  than  the 
sole  source  firm,  ample  ex¬ 
perience  for  achieving  a  com¬ 
petitive  position 

A  performance-cost  trade-off 
allows  the  second  firm  to 
reduce  his  cost  by  providing 
service  with  lower  perform¬ 
ance  levels  than  those  pro¬ 
vided  by  the  sole  source  firm, 
but  performance  levels  which 
still  meet  government  require¬ 
ments 

System  modifications  result  in 
a  new  cost  improvement  situa¬ 
tion  for  both  firms . 


POTENTIAL  APPLICATIONS 

The  authors  have  developed  a  unique 
framework  in  which  some  basic  principles 
of  service  contracting  were  analyzed  and 
the  effects  of  competition  were  examined. 
The  central  feature  of  the  authors'  work 
is  a  shift  and  rotation  of  the  sole  source 
cost  improvement  curve  in  the  presence  of 
competition.  This  phenomenon  represents 
an  immediate  reduction  in  the  current 
level  of  costs  (shift)  and  an  acceleration 
in  the  rate  of  cost  improvement  (rotation) 
which  applies  to  future  periods  of  per¬ 
formance.  The  magnitude  of  these  parame¬ 
ters  depends  upon  the  amount  of  competi¬ 
tive  pressure  which  a  second  firm  applies 
on  the  sole  source  firm. 

When  transformed  into  a  computer- 
based  model  and  used  along  with  an  accom¬ 
panying  data  base,  the  framework  is  appli¬ 
cable  to  specific  service  programs  for  the 
purposes  of  forecasting  savings  (losses) 
from  competition.  The  model  provides  a 
useful  tool  for  program  management  and 
supplies  information  which  helps  managers 
determine  the  future  costs  of  their  pro¬ 
gram,  the  cost-effectiveness  of  competi¬ 
tion,  and  the  requirements  for  a  success¬ 
ful  competition.  Sensitivity  analyses  can 
be  performed  on  key  factors  such  as  the 
timing  of  competition,  performance  re¬ 
quirements,  and  the  cost  improvement  rate. 

Each  application  of  the  model  is 
individually  constructed.  The  nature  and 
background  of  the  specific  program  are 
researched.  Data  are  gathered  on  similar 
programs  and  a  probable  range  of  parameter 
values  are  determined  for  key  variables 
such  as  the  sole  source  cost  improvement 


rate,  and  the  shift  and  rotation  of  the 
sole  source  curve. 

Information  and  data  which  is  collec¬ 
ted  and  analyzed  prior  to  application  of 
the  model  includes,  the  following: 

•  Cost,  contract,  and  general 
history  of  the  program 

•  Past  and  future  changes  in 
the  scope  of  the  work 

•  Contractor  performance  reports 

•  Cost,  contract,  and  perform¬ 
ance  reports  on  similar  programs 

•  Technical  data  requirements  (e.g., 
cost  of  a  data  package) 

•  The  nature  of  the  industry  pro¬ 
viding  the  service. 

It  is  important  to  note,  that  continual 
development  of  this  analytical  framework 
is  occurring.  Each  application  provides 
new  insights  and  additional  data  which  are 
used  in  further  construction  and  refine¬ 
ment  of  the  model  for  the  purposes  of  help¬ 
ing  managers  recognize  and  capture  poten¬ 
tial  cost  savings  on  their  service  pro¬ 
grams  . 
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COST-EFFECTIVENESS  OF  MAINTENANCE 
SIMULATORS  FOR  MILITARY  TRAINING 

Jesse  Orlansky  and  Joseph  String 
institute  For  Defense  Analyses 
Arlington,  Virginia 

ABSTRACT 


The  cost-effectiveness  of  maintenance  simulators  is  compared  to  that  of 
actual  equipment  trainers  for  training  military  maintenance  technicians.  Main¬ 
tenance  simulators  are  as  effective  as  actual  equipment  trainers  when  measured 
by  student  achievement  at  scnool;  there  is  no  difference  in  the  job  performance 
of  students  trained  either  way,  according  to  supervisors'  ratings  (based  on  one 
study).  The  acquisition  cost  of  maintenance  simulators  is  less  than  that  of 
actual  equipment  trainers;  they  cost  Jess  than  60  percent  as  much  if  develop¬ 
ment  costs  are  included  and  less  than  20  percent  as  much  if  only  unit  fabrica¬ 
ting  costs  are  considered.  Acquisition  and  use  of  a  maintenance  simulator  over 
a  15-year  period  would  cost  38  percent  as  much  as  an  actual  equipment  trainer 
(according  to  one  life-cycle  cost  comparison).  Since  maintenance  simulators 
and  actual  equipment  trainers  are  equally  effective  and  since  maintenance  simu¬ 
lators  cost  less,  it  is  concluded  that  maintenance  simulators  are  more  cost- 
effective  than  actual  equipment  trainers.  This  finding  is  qualified  because 
it  comes  from  a  limited  number  of  comparisons,  because  effectiveness  is  based 
primarily  on  school  achievement  rather  than  on-the-job  performance  and  because 
it  is  based  primarily  on  acquisition  rather  than  on  life-cycle  corts. 


INTRODUCTION 

This  paper  compares  the  cost- 
effectiveness  of  maintenance  training 
simulators  and  actual  equipment  trainers 
for  use  in  training  military  personnel  how 
to  maintain  operational  equipment.  Both 
types  of  equipment  have  been  used  for 
training  personnel  to  perform  corrective 
and  preventive  maintenance  at  organiza¬ 
tional  and  intermediate  levels  (Orlansky 
and  String  1981). 

Actual  equipment  trainers  have  long 
been  used  in  technical  training  schools 
for  two  significant  reasons:  (1)  they  can 
be  acquired  simply  by  ordering  additional 
units  of  operational  equipment  already 
being  procured  as  components  of  weapon  and 
support  systems;  and  (2)  they  provide  real¬ 
istic  training  on  the  equipment  to  be  main¬ 
tained  after  leaving  school.  Operational 
equipment  can  be  modified  for  training  by, 
for  example,  placing  it  on  a  stand  and 
adding  power  supplies,  input  signals  and 
controls  needed  to  make  it  operate  in  a 
classroom.  In  recent  years,  there  has  been 
a  trend  to  use  maintenance  training  simu¬ 
lators  rather  than  actual  equipment  for 
training  purposes.  Maintenance  simulators 
are  said  to  have  advantages  for  use  in 
training  such  as  lower  cost,  ability  to 
demonstrate  ?.  wider  variety  of  malfunctions 
and  more  freedom  from  breakdown  in  the 
classroom. 

MAGNITUDE  OF  THE  PROBLEM 

Maintenance  is  a  critical  aspect  of 
defense  planning  and  operations  and  costs 
$18-20  billion  e.-'ch  year,  including  the 
costs  of  spare  parts,  supplies  and  modifi¬ 
cations  (Turke  1977,  p.  5).  According  to 
the  General  Accounting  Office,  the  Army 
spends  25  percent  ($7  billion  in  FY  1978) 
of  its  annual  budget  on  maintenance;  over 


200,000  mechanics  and  equipment  operators 
in  the  Army  have  specific  unit-level  main¬ 
tenance  responsibilities  (GAO  1978,  p.  1). 
In  the  Air  Force,  maintenance  requires 
about  28  percent  of  the  work  force  (mili¬ 
tary  and  civilian)  and  costs  between  $5 
and  $7  billion  annually  (Townsend  1980), 
Labor  for  repairs  is  estimated  to  account 
for  39  percent  of  the  cost  of  recurring 
logistical  support  of  the  Air  Force  A-7D 
aircraft  (Fiorello  1975).  Specialized 
skill  training  at  military  schools  will 
cost  about  $3.4  billion  or  33  percent  of 
the  cost  of  individual  training  in  fiscal 
year  1982  (Department  of  Defense,  Military 
Manpower  Training  Report  for  FY  1982,  p.6); 
the  portion  attributed  solely  to  mainte¬ 
nance  training  is  not  known.  The  cost  of 
on-the-job  training,  that  follows  school 
training,  is  also  not  known. 

The  three  services  spent  over  $5  mil¬ 
lion  in  FY  1979  for  research  and  develop¬ 
ment  on  maintenance  simulators.  About  $3.7 
million  (68  percent)  of  these  funds  (cate¬ 
gory  6.4  funds)  were  for  the  development 
and  procurement  of  prototype  training 
equipment.  About  30  different  maintenance 
simulators  were  either  under  contract  or 
planned  for  development,  as  of  February 
1981. 

There  are  now  about  3600  different 
types  of  maintenance  training  devices  in 
the  Air  Force  to  support  aircraft  systems. 
The  Air  Force  Air  Training  Command  esti¬ 
mates  that  the  current  inventory  of  all 
maintenance  training  devices  cost  $500 
million,  of  which  $350  million  is  for  air¬ 
craft  maintenance  alone  (Aeronautical 
Systems  Division,  1978).  The  procurement 
of  maintenance  simulators  for  the  F-16  air¬ 
craft  is  estimated  to  cost  about  $32  mil¬ 
lion,  including  some  units  to  be  delivered 
to  NATO  countries. 
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One  large  Industrial  contractor  has 
estimated  that  the  Department  of  Defense 
will  spend  over  $600  million  for  mainte¬ 
nance  trainers  from  1977  to  1985;  annual 
procurements  are  estimated  to  reach  about 
$120  million  per  year  by  1984  (Figure  1), 


The  distribution  of  this  procurement, 
according  to  type  of  simulator,  is  Bhown 
in  Figure  2.  Outside  the  United  States, 
the  procurement  of  maintenance  simulators 
is  estimated  to  be  about  $5.5  million  par 
year. 


The  "Electronics-X"  study,  conducted 
in  1974,  was  a  major  effort  to  determine 
the  cost  and  reliability  of  military  elec¬ 
tronic  equipment  (Gates,  Gourary,  Deitch- 
man,  Rowan  and  Weimer,  1974).  Four  methods 
were  used  to  estimate  the  cost  of  maintain¬ 
ing  electronics  equipment  each  year.  The 
results  ranged  f ran  $3.4  billion  to  $6.8 
billion  with  an  average  of  $5.4  billion  per 
year  (Gates,  Gourary,  Daitchraan  et  al., 

1974,  Vol.  II,  p.  374).  The  estimate  of 
$5.4  billion  per  year  for  maintenance  is 
about  equal  to  the  cost  of  procuring  elec¬ 
tronic  equipment  each  year  (Gates,  Gourary, 
Deitchman  et  al , ,  1974,  Vol.  I,  p.  52). 

Note  that  procurement  costs  relate  to 
acquiring  current  technology)  the  mainte¬ 
nance  costs  relate  to  systems  whose  average 
age  is  about  ten  years. 

The  costs  for  manpower  were  estimated 
by  the  Defense  Science  Board  Task  Force  on 
Electronics  Management  to  account  for 
perhaps  &s  much  as  75  percent  of  the  costs 
of  maintaining  military  electronics  equip¬ 
ment;  actual  costs  are  unknown  due  to 
limitations  in  the  cost  allocation  system 
(DSb,  1974,  p„  14). 

CHARACTERISTICS  OF  MAINTENANCE  SIMULATORS 

Maintenance  simulators  now  under 
development  differ  notably  in  their  resem¬ 
blance  to  actual  equipment,  their  ability 
to  provide  instructional  services,  and  in 
their  complexity  and  cost.  These  simula¬ 
tors  are  often  characterized  as  2-D  or  3-D 
devices,  i.e.  ,  aa  being  two-  or  three- 
dimensional  in  their  physical  form;  some 
simulators  contain  both  2-D  and  3-D  compo¬ 
nents. 

The  manufacturers  of  2-D  simulators 
have  developed  software  packages,  computer 
and  support  equipment  that  can  be  used  in 
a  number  of  different  simulations.  This 
has  led  us  to  distinguish  between,  what  we 
call  later  in  discussing  costs,  "standard" 
and  "non-standard"  maintenance  simulator 
systems-  standard  systems,  whether  they 
are  2-D  or  3-D  simulators,  are  likely  to 
cost  less  than  non-standard  systems.  A  3-D 
simulator  permits  "hands  on"  practice  in 
manual  maintenance  skills  not  possible  on 
many  2-D  simulators;  it  also  has  greater 
physical  similarity  to  the  actual  equip¬ 
ment.  Whether  or  not  greater  physical 
similarity  increases  the  effectiveness  of 
training  is  a  proper  question. 

Advantages  of  Maintenance  Simulators 

The  advantages  of  simulators  for 
training  maintenance  personnel  have  been 
recognized  for  many  years  (e.g.,  R.B. 

Miller  1954,  Gagne  1962,  Lumsdaine  1960, 
val verde  1968,  Kinkade  and  Wheaton  1972, 
G.G.  Miller  1974,  Monteraerlo  1977,  and  Fink 
and  Shrivcr  1978).  The  major  advantage  of 
a  maintenance  simulator  is  that,  as  a 
training  device,  it  can  be  designed  to  pro¬ 
vide  facilities  important  for  instructing 
students?  in  contrast,  actual  equipment  is 
designed  to  perform  some  military  function 
and  is  not  intended  to  be  a  training 
device. 


Maintenance  simulators  can  be  designed 
to  demonstrate  a  large  variety  of  malfunc¬ 
tions  with  which  maintenance  personnel 
should  be  familiar,  including  those  that 
cannot  be  demonstrated  conveniently  on 
actual  equipment  trainers  or  that  occur 
rareiy  in  real  life.  All  modern  mainte¬ 
nance  simulators  incorporate  some  type  of 
computer  support.  Thus,  the  symptoms  of 
many  types  of  complex  faults  can  be  stored 
in  the  computer  and  selected  simply  by  a 
control  setting  on  the  instructor’s  con¬ 
sole.  computer-supported  equipment  can 
also  record  what  the  student  does,  thereby 
reducing  the  need  for  constant  observation 
by  the  instructor.  The  instructor  can  use 
information  collected  by  the  computer  to 
guide  each  student;  a  computer  can  also 
assist  the  student  without  an  instructor’s 
intervention.  Records  of  student  perfor¬ 
mance  and  achievement  can  be  maintained 
automatically.  Simulators  can  be  made 
rugged  enough  to  sustain  the  damage  or 
abuse  encountered  from  students.  Tnus, 
they  can  provide  greater  reliability  and 
availability  in  the  classroom  than  is  often 
often  possible  with  actual  equipment. 
Training  that  would  be  avoided  because  of 
safety  reasons,  e.g.  ,  exposure  of  ntudents 
co  dangerous  electrical  currents  or  hydrau¬ 
lic  pressures,  can  be  undertaken  with 
little  risk  with  a  simulator.  If  students 
using  such  equipment  complete  their  train¬ 
ing  in  lese  time,  as  has  often  been  found 
with  computer-based  methods  of  instruction, 
there  are  potential  cost  benefits  due  to 
savings  in  student  time,  increased  through¬ 
put  of  students  and  reduced  need  for 
instructors  and  support  personnel. 

A  simulator  need  not  contain  all 
the  components  found  in  the  actual  equip¬ 
ment.  Thus,  it  is  often  possible  to  build 
a  simulator  that  has  greater  flexibility 
and  capacity  for  training  and  costs  less 
than  an  actual  equipment  trainer. 

Disadvantages  of  Maintenance  Simulators 

There  are  some  disadvantages  to  the 
use  of  simulators.  7hfc  procurement  of 
maintenance  simulators  necessarily  involves 
costs  to  design  and  build  this  special 
equipment,  to  develop  course  materials, 
maintenance  procedures,  support  and  docu¬ 
mentation.  The  types  of  training  provided 
by  simulators  may  not  provide  the  student 
with  all  the  skills  needed  to  maintain 
operational  equipment,  an  outcome  that 
seems  assured  when  actual  equipment  is 
used  for  training.  A  simulator  mey  not  be 
ready  when  needed  for  training  the  initial 
cadres  of  a  new  weapon  system  because  its 
design  and  development  requires  some  effort 
in  addition  to  or  at  least  parallel  to  that 
needed  for  the  actual  equipment  which  is 
already  being  produced  for  the  new  system; 
modifications  in  the  design  cf  the  actual 
equipment  for  a  new  system  may  also  require 
modifications  in  the  simulator  and  delay 
its  delivery.  If  there  are  many  and  fre¬ 
quent  modifications  to  the  system,  the 
original  simulator  may  have  to  be  rede¬ 
signed  totally  at  some  additional  cost,  in 
order  to  be  useful  for  training. 
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Data  on  the  effectiveness  and  cost 
of  maintenance  simulators  and  actual 
equipment  trainers  are  considered  next. 

THE  EFFECTIVENESS  OF 
MAINTENANCE  SIMULATORS 

The  purpose  of  maintenance  training, 
whether  with  simulators  or  actual  equip¬ 
ment  trainers,  is  to  qualify  technicians 
to  maintain  equipment  in  the  field.  In 
fact,  however,  the  effectiveness  of  mainte¬ 
nance  simulators  for  training  technicians 
has  been  compared  to  that  of  actual  equip¬ 
ment  trainers  only  on  the  basis  of  student 
performance  at  school  and  not  on  the  job? 
there  is  one  exception  to  this  general 
statement  (Cicch inel li ,  Harmon,  Keller  and 
Kottenstette,  1980).  The  lack  of  job  per¬ 
formance  data  to  validate  training  applies 
generally  to  all  types  of  military  training 
rather  than  to  maintenance  training  alone. 

Effectiveness  of  Maintenance  Simulators 
at  Schools 

We  found  12  studies,  conducted  over 
the  period  of  1967  to  1980,  that  compare 
the  effectiveness  of  maintenance  simulators 
and  actual  equipment  trainers  for  training 
in  a  variety  of  courses  at  military  train¬ 
ing  schools;  these  are  summarized  in 
Figure  3. 

Most  of  the  maintenance  simulators 
apply  to  electronics  and  aviation,  e.g., 
radar,  propellers,  engines,  flight  con¬ 
trols,  FM  tuner,  test  equipment;  one,  the 
Hagen  Automatic  Boiler  Control,  involves 
an  electro-mechanical  system  for  ships. 

The  training  segments  or  courses  in  which 
they  were  used  varied  in  length  from  3 
hours  to  5  weeks  (median  4.7  days,  N  =  12 
courses);  the  number  of  subjects  trained 
with  simulators  in  these  courses  varied 
from  6  to  56  (median  16,  N  *  14  groups);  a 
grand  total  of  267  students  was  involved 
in  all  of  these  studies. 

Student  Achievement 

Effectiveness  was  evaluated  by  com¬ 
paring  the  scores,  in  end-of-course  tests, 
of  students  who  used  simulators  with  those 
who  used  actual  equipment  trainers.  There 
are  13  comparisons;  in  12  of  these,  stu¬ 
dents  trained  with  simulators  achieved  the 
same  or  better  test  scores  than  those 
trained  with  actual  equipment;  in  one  case, 
scores  were  lower.  The  differences,  though 
statistically  significant,  were  small. 

Cicchinelli,  Harmon,  Keller  and 
Kottenstette  (1980)  compared  supervisors' 
ratings  on  the  job  performance  of 
technicians  trained  either  with  a 
maintenance  simulator  (the  6883  Test 
Station  3-D  Simulator)  or  an  actual 
equipment  trainer.  Field  surveys  provided 
data  on  the  job  performance  of  course 
graduates  (some  twice),  some  of  whom 
were  on  the  job  as  long  as  32  weeks. 

The  supervisors  did  not  know  how  the 
students  had  been  trained.  Their  ratings 
showed  no  noticeable  difference  between 


the  performance  of  technicians  trained 
with  the  simulator  or  the  actual  equipment 
trainer.  The  abilities  of  the  technicians 
in  both  groups  increased  with  amount  of 
time  on  the  job. 

Time  Savings 

The  automated  and  individualized 
method  of  instruction  that  is  an  inherent 
characteristic  of  modern  maintenance  simu¬ 
lators  should  be  expected  to  save  some  of 
the  time  students  need  to  complete  the  same 
course  when  given  by  conventional  instruc¬ 
tion  (orlansky  and  String  1979).  Such  time 
savings  are  reported  in  three  of  these 
studies  (Parker  and  DePauli,  1967,  Rigney, 
Towne,  King  and  Moran,  1978  and  Swezey, 
1979);  compared  to  the  use  of  actual  equip¬ 
ment  trainers,  maintenance  simulators 
saved  22,  50  and  50  percent,  respectively, 
of  the  time  students  needed  to  complete 
these  courses.  Although  no  explanations 
are  offered  for  these  time  savings,  one 
could  surmise  that  they  are  due  to  such 
factors  as  that  brighter  students  can 
complete  a  self-paced  course  faster  than 
one  given  by  conventional,  group-paced 
instruction,  that  maintenance  simulators 
generally  have  greater  reliability  in  the 
classroom  than  do  actual  equipment  trainers 
and  that  instructors  need  less  time  to  set 
up  training  problems  and/or  to  insert  mal¬ 
functions  in  simulators  than  in  actual 
equipment  trainers. 

Attitudes 

Based  on  questionnaires  administered 
at  the  completion  of  the  courses,  students 
favor  the  use  of  simulators  in  9  of  10 
cases  and  are  neutral  in  one.  Instructors 
are  equally  divided  (about  one-third  in 
each  category  of  response)  in  being  favor¬ 
able  ,  neutral  or  unfavorable  to  the  use  of 
simulators. 

Relevant  Data  from  Computer-Based 
Instruction 

Modern  maintenance  simulators  can  pro¬ 
vide  individualized  instruction  on  a  series 
of  prescribed  lessons.  They  can  also  mea¬ 
sure  student  performance  and  see  that  the 
student  does  not  go  to  a  new  lesson  until 
he  has  mastered  the  preceding  ones.  The 
instructional  strategies  employed  in  these 
simulators  are  derived  from  widely  used 
methods  of  instruction  called  computer- 
assisted  and  computer-managed  instruction; 
both  are  individualized  and  self-paced  in 
nature  and  use  computers  to  monitor  student 
progress.  In  computer-assisted  instruction 
(CAI ) ,  all  the  instructional  material  is 
stored  in  a  computer  and  presented  to  the 
student  in  a  controlled  manner  via,  e.g.  ,  a 
cathode  ray  tube  or  a  visual  projection 
device  with  random  access  to  a  large  reser¬ 
voir  of  slides.  The  student  responds  to 
this  material  by  touching  portions  of  the 
screen  sensitive  to  touch  or  by  using  a 
keyboard  or  teletypewriter.  In  computer- 
managed  instruction  (CMI),  the  lessons  are 
performed  away  from  the  computer  in  a 
learning  carrel  or  on  a  laboratory  bench 
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SIMULATOR  TO  ACTUAL  EQWPMENT 


FIGURE  3.  Summary  of  Studies  on  the  Effectiveness  of  Maintenance  Simulators,  1967-1980 


setup.  The  student  takes  a  test  at  the 
completion  of  each  lesson;  the  answers,  on 
a  sheet,  are  scored  by  the  computer  via  an 
optical  reader,  which  then  directs  the  stu¬ 
dent  to  a  new  lesson  or  to  additional  prac¬ 
tice  on  the  current  one. 

CAI  and  CMI  systems  are  not  mainte¬ 
nance  simulators  but  they  have  been  used  to 
provide  certain  aspects  of  maintenance 
training,  e.g.,  knowledge  of  operating 
principles,  troubleshooting  procedures, 
fault  identification,  and  the  knowledge 
aspects  of  remove  and  replace  actions 
(i.e.,  what  the  technician  should  do  after 
a  fault  is  identified  rather  than  perform 
the  task  with  actual  parts).  Knowledge 
about  maintenance  procedures  can  be 
acquired  on  a  CAI  and  CMI  system  but  this 
is  accomplished  with  less  fidelity  and 
with  little  of  the  hands-on  experience  than 
can  be  provided  by  a  maintenance  simulator, 
particularly  of  the  3-D  variety.  Some  of 
the  new  maintenance  simulators  are  essen¬ 
tially  CAI  systems. 

Student  Achievement 

In  a  previous  study,  the  authors  exam¬ 
ined  tne  cost-effectiveness  of  computer- 
based  instruction  in  military  training 
(Orlansky  and  String  1979).  Some  of  the 
courses  on  which  effectiveness  data  were 
available  involved  instruction  similar  to 
that  provided  on  maintenance  simulators, 
i.e.,  basic  electronics,  vehicle  repair, 
fire  control  system  maintenance,  precision 
measuring  equipment  and  weapons  mechanic. 
Data  on  student  achievement  in  these 
courses  are  presented  in  Figure  4;  there 
are  28  comparisons  of  conventional  instruc¬ 
tion  with  CAI  and  two  with  CMI.  Student 
achievement  in  these  courses  at  school  with 
CAI  or  CMI  was  the  same  as  or  superior  to 
that  provided  by  conventional  instruction? 
the  amount  of  superior  performance,  when 
present,  was  small.  This  is  consistent 
with  what  we  found  for  maintenance  simula¬ 
tors. 

Time  Savings 

Data  on  the  amount  of  student  time 
saved  by  CAI  and  CMI  in  these  courses, 
compared  to  conventional  instruction,  are 
shown  in  Figure  5;  there  are  30  compari¬ 
sons.  The  amount  of  time  saved  by 
computer-based  instruction  varied  from 
-32  to  59  percent  with  a  median  value  of 
28  percent. 

THE  COST  OF 
MAINTENANCE  SIMULATORS 

Many  people  believe  that  the  cost  of  a 
maintenance  simulator  is  a  function  of  the 
fact  that  it  is  a  two-dimensional  or  three- 
dimensional  device.  There  is  a  certain 
plausibility  to  this  point  of  view  which 
relates  the  physical  characteristics  and 
complexity  of  a  simulator  to  its  cost.  But 
another  important  cost  factor  concerns  the 
number  of  units  that  are  procured  and, 
thus,  the  average  cost  of  each  unit.  In 


order  to  deal  with  the  issue  of  costs,  we 
divided  simulators  into  three  classes  called 
standard,  non-standard  and  CAI-like 
systems. 

Standard  Systems 

This  class  of  maintenance  simulators 
is  based  on  standardization  of  the  physical 
configuration.  Such  simulators  consist  of 
two  elements:  one  element,  called  here  the 
"general  simulation  system"  constitutes  a 
generalized  and  adaptable  (but  incomplete) 
simulation  capability  that  can  satisfy  a 
wide  range  of  specific  training  applica¬ 
tions.  The  second  clement,  that  tailors 
the  general  simulation  system  to  a  particu¬ 
lar  training  application,  is  typically  limi¬ 
ted  to  courseware  and  pictorial  or  other 
representations  (i.e.,  the  simulation 
model)  of  the  particular  equipment  being 
simulated.  Standard  systems  were  the  ear¬ 
liest  type  to  be  used  for  maintenance 
training  and  are  the  only  class  to  achieve 
extensive  use.  compared  with  the  other 
classes  of  simulators,  the  standard  systems 
are  generally  low  in  cost  and  limited  in 
terms  of  the  complexity  of  processes  that 
can  be  simulated.  About  650  units  of  stan¬ 
dard  simulators  have  been  procured  for 
about  200  different  training  applications 
(most  produced  by  ECC,  Burtek,  Ridgeway, 
and  Lockheed ) . 

Non-Standard  Systems 

The  outstanding  characteristic  of 
non-standard  systems  is  diversity,  encom¬ 
passing  different  contractors  and  types  of 
contracts,  program  purpose,  numbers  of 
devices  manufactured,  physical  character¬ 
istics,  complexity,  and  cost. 

The  physical  characteristics  of  the 
non-standard  simulators  are  diverse  and 
include  two-  and  three-dimensional 
trainers.  There  is  wide  variability  in 
the  software.  Further,  since  most  non¬ 
standard  systems  typically  simulate  only 
one  operational  system,  there  is  no  defini¬ 
tive  separation  between  software  and 
courseware  functions.  There  are  now  about 
17  non-standard  maintenance  simulator  pro¬ 
grams  that  will  produce  687  units  of  47 
unique  maintenance  simulators,  e.g.  ,  the 
Mk  92  Fire  Control  System,  Close-In  Weapon 
System,  F-16,  MA-3  and  6883  Test  Bench, 
producers  of  these  simulators  include 
Honeywell,  Vought,  Applimation,  Grumman 
and  RCA. 

CAI-Like  Systems 

A  CAI-like  maintenance  simulator  is  a 
computer-assisted  instruction  (CAI)  system 
with  courseware  designed  specifically  to 
train  maintenance  skills.  A  typical  CAI 
system  uses  a  two-dimensional  display 
(cathode  ray  tube  and/or  random  access 
slide  or  microfiche  projector)  to  present 
lesson  materials  (pictures  of  equipment 
and  the  like)  under  control  of  a  computer 
that  also  monitors  student  progress, 
prescribes  lessons,  and  scores  tests.  When 
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FIGURE  5.  Amount  of  Student  Time  Saved  in  Courses  Relevant 
CAI  and  CMI  Compared  to  Conventional  Instruction 
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adapted  to  maintenance  training ,  the  CAI 
features  are  retained,  and  tne  trainer  may 
also  employ  three-dimensional  versions  of 
equipment.  Examples  of  such  systems  are 
the  Navy  Electronic  Equipment  Maintenance 
Trainer  and  the  Army  Maintenance  Training 
and  Evaluation  Simulation  Svstem.  insuf¬ 
ficient  cost  data  were  available  on  CAI- 
like  maintenance  simulators  and  they  are 
not  discussed  further. 

Costs  of  Maintenance  Training  simulators 

We  learned,  to  our  regret,  that  the 
data  now  available  on  standard  systems  are 
insufficient  to  analyze  their  elements  of 
cost  and  to  relate  these  cost  elements  to 
the  physical  and  performance  characteris¬ 
tics  of  the  trainers.  In  effect,  it  is  now 


difficult  or  impossible  to  identify  the 
major  cost  distinctions  (e.g.,  between 
recurring  and  non-recurring  costs,  between 
development  and  fabrication,  between  hard¬ 
ware  and  software)  that  allow  characteris¬ 
tics  of  the  simulator  to  be  related  to  the 
total  cost  of  the  simulator  program. 

Data  from  nine  contracts  for  standard 
simulators  were  reviewed,  and  the  informa¬ 
tion  they  contain  is  shown  in  Figure  6. 
These  contracts  involve  the  development  of 
67  different  models  of  simulators  and  the 
delivery  of  a  total  of  444  units.  The 
figure  shows  average  contract  cost  per 
delivery  (total  contract  value  divided  by 
the  number  of  trainers  procured)  vs  the 
number  of  -  trainers  procured  in  each  con¬ 
tract.  These  simulators  ranged  in  unit 
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FIGURE  6. 


cost  from  about  $10  thousand  to  $204  thou¬ 
sand  each,  with  a  median  cost  of  about 
$33 ,000.  As  we  would  expect,  the  unit  cost 
is  reduced  as  the  number  of  units  in  each 
contract  increases.  However,  these  simula¬ 
tors  arc  not  a  homogenous  sample;  they  vary 
in  their  complexity,  physical  and  perfor¬ 
mance  characteristics.  Therefore,  caution 
is  advised  in  using  the  data  in  this  fig¬ 
ure. 

The  cost  of  13  non-standard  mainte¬ 
nance  simulators  is  shown  in  Figure  7.  The 
estimates  are  normalized  to  show  recurring 
production  costs  adjusted  to  reflect  a  pro¬ 
duction  quantity  of  one;  costs  of  develop¬ 
ment  and  test  are  not  included.  These 
simulators  range  in  cost  from  $100  thousand 
to  $4.5  million;  the  median  value  is  $900 
thousand . 

The  non-recurring  costs  account  for  a 
large  portion  of  the  total  program  costs  of 
non-standard  maintenance  simulators — over 
70  percent  when  only  unit  is  fabricated  and 
about  50  percent  when  five  or  six  are  fab¬ 
ricated  (Figure  8).  Software  and  course¬ 
ware  account  for  10  to  45  percent  of  total 
program  costs  (Figure  9). 

COST-EFFECTIVENESS  OF 

MAINTENANCE  SIMULATORS 

He  found  that  student  achievement  at 
school  is  about  the  same  whether  students 
are  trained  with  maintenance  simulators  or 
with  actual  equipment  trainers.  Therefore, 
the  relative  cost-effectiveness  of  mainte¬ 
nance  simulators  and  actual  equipment 
trainers  depends  on  how  much  each  costs. 


We  have  just  shown  what  maintenance 
simulators  cost;  next,  we  must  compare  the 
costs  of  simulators  and  of  actual  equipment 
trainers.  But  note  that  the  data  on  main¬ 
tenance  simulators  refer  only  to  procure¬ 
ment  costs.  These  data  do  not  include  the 
coats  of  using  these  simulators,  such  as 
for  instructors,  student  pay,  support  and 
travel,  maintenance  of  the  training  equip¬ 
ment  and  management  of  the  school.  There 
is  one  life-cycle  cost  comparison  that  we 
will  consider  separately.  The  cost  com¬ 
parison  that  follows  is  incomplete  because 
it  is  based  only  on  acquisition  costs. 

The  cost  of  an  actual  equipment 
trainer  is  the  production  cost  of  one 
unit  of  equipment  under  procurement  for 
some  military  system;  this  value  does  not 
include  the  costs  of  research,  development, 
test  and  evaluation  (RDT&E).  Adapting  a 
component  of  an  operational  system  for  use 
in  training,  such  as  by  adding  power, 
special  inputs  and  controls,  may  require 
some  additional  costs  attributable  to 
training. 

We  were  able  to  get  relatively  com¬ 
plete  data,  useful  for  comparative  pur¬ 
poses,  on  both  maintenance  simulators  and 
actual  equipment  trainers,  for  only  11 
cases;  comparisons  were  not  possible  for 
some  recently  developed  maintenance  simula¬ 
tors  where  actual  equipment  trainers  had 
not  been  used  previously  for  training. 

Some  of  the  simulators  are  prototypes, 
rather  than  production  units;  data  on 
these  simulators  include  the  costs  of 
research  and  development.  The  costs  of 
research  and  development  should  be  removed 


Trainer 

Cost 

$(000) 

AN/TPS-43  Ground  Radar 

100 

Trident  Air  Conditioner 

135 

Trident  High  Pressure  Air  Compressor 

140 

F-lllD  Avionics  Test  Bench  (2-D  6883) 

39  5 

A-6E  TRAM 

475 

MA-3  Generator/Constant  Speed  Drive 

Test  stand 

525 

AWACS  Radar  System 

900 

F-lllD  Avionics  Test  Bench  (3-D  6883) 

920 

A-7E  Heads-Up  Display  Test  Bench 

1295 

F-4J/N  (AT  Trainer) 

1540 

AWACS  Navigation/Guidance  System 

2460 

Trident  Integrated  Radio  Room  -  Maintenance 
Trainer 

2625 

Trident  Integrated  Radio  Room  -  Operator/ 
Maintenance  Trainer 

4465 

FIGURE  7.  Acquisition  Costs  of  13  Non-Standard  Maintenance 
Simulators  (Normalized  to  Include  Recurring  Costs 
for  a  Production  Quantity  of  1) 
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FIGURE  8.  Non-Recurring  Cost  as  a  Percent  of  Program  Total 
Cost  According  to  Quantity  Fabricated 
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FIGURE  9. 


Software/Courseware  Cost  as  a  Percent  of  Program 
Total  Cost,  According  to  Quantity  Fabricated 

456 


in  order  to  make  a  fair  comparison  of  main¬ 
tenance  simulators  with  actual  equipment 
trainers  which,  as  noted  above,  are  pro¬ 
duction  items  and  exclude  such  costs. 

The  number  of  maintenance  simulators  pro¬ 
cured  could  also  influence  the  cost  of  a 
single  unit;  this  varied  from  1  to  36. 

We  decided  to  use  estimates  which 
would  bracket  the  cost  of  one  maintenance 
simulator  within  high  and  low  limits. 

These  were? 

High  cost  estimate?  Total  production 
costs  adjusted  to  reflect  a  production 
quantity  of  one;  this  includes  the 
costs  of  research  and  development  but 
not  of  test  and  evaluation.  We  call 
this  the  "Simulator  Normalized  Program 
Cost", 

Low  cost  estimate?  The  cost  of  pro- 
ducing  a  £ollow-on  maintenance  simula¬ 
tor  after  the  costs  of  RDT&E ;  proto¬ 
types  and  manufacturing  facilities 
have  been  accounted  for.  We  call  this 
the  "Simulator  Unit  Recurring  Fabri¬ 
cation  Cost." 


The  high  cost  estimates  are  shown  in 
Figure  10.  The  ratio  of  simulator/actual 
equipment  trainer  costs  is  0.60  or  less 
for  seven  cases  cases  (range  0.25  to. 0.55). 
There  are  four  cases  where  this  ratio 
varies  from  1.60  to  4.00  (VTAS,  MA-3,  AT 
Trainer  and  AWACS).  We  believe  these  data 
are  suspect  for  one  or  more  of  the  follow¬ 
ing  reasons?  the  costs  of  the  operational 
equipments  (some  of  which  are  relatively 
old)  may  have  been  considerably  under¬ 
estimated;  the  costs  of  the  simulators, 
some  of  which  are  designed  for  use  in 
research,  may  be  high  because  they  include 
capabilities  not  needed  for  routine  train¬ 
ing.  For  these  reasons,  we  decided  to 
accept  0.60  as  an  upper  limit  for  the  rela¬ 
tive  cost  of  a  maintenance  simulator  com¬ 
pared  to  an  actual  equipment  trainer. 

The  low  cost  estimates,  based  on  the 
recurring  cost  of  these  simulators,  are 
shown  in  Figure  11.  Nine  of  the  11  cases 
fall  at  0.20  or  lower;  the  range  is  0.03 
to  0.19.  The  two  outlyers  (VTAS  and  MA-3) 
are  regarded  as  atypical  for  the  reasons 
set  forth  above. 


19  100  1,000  10,000  100,000 

ACTUAL  EQWPMENT  TRAINER  COST  (thwiundt  $1  datars) 

a-ia-ai-at 


Relation  Between  Actual  Equipment  Trainer  and  Simulator  Normalized 
Program  Costs 
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FIGURE  10. 


10  100  1000  10,000  100,000 

ACTUAL  EQUMENT  TRAMER  COST  (thftuundt  at  dobra) 

I-1M1-I1 

FIGURE  II.  Relation  Between  Actual  F.qu1pment  Trainer  and  Simulator  Recurring 
Fabrication  Costs 


We  conclude,  therefore,  that  the 
acquisition  costs  of  simulators  generally 
fall  in  the  range  of  20  to  60  percent  that 
of;  actual  equipment  trainers.  These  are 
very  conservative  estimates. 

The  cost-effectiveness  of  a  mainte¬ 
nance  simulator  on  a  life-cycle  basis  has 
been  evaluated  only  in  one  case,  that  of 
the  Air  Force  6883  Test  Stand  3-Dimensional 
Simulator  and  6883  Actual  Equipment  Trainer 
(Cicch inelii ,  Harmon,  Keller  and  Kotten- 
stette,  1980).  The  three-dimensional 
simulator  and  actual  equipment  trainer  were 
equally  effective  when  measured  by  student 
achievement  at  school?  supervisors'  ratings 
showed  no  difference  between  the  job  per¬ 
formance  of  students  trained  either  way  for 
periods  up  to  32  weeks  of  experience  after 
leaving  school. 

The  life-cycle  cost  comparison  of 
simulator  and  actual  equipment  trainer  is 
shown  in  Figure  12.  Costs  were  estimated 
in  constant  1978  dollars  over  a  15-year 
period  and  discounted  at  10  percent.  The 
results  show  that  the  total  cost  per  stu¬ 
dent  hour  was  $23  for  the  simulator  and 
$60  for  the  actual  equipment  trainer,  i.e.. 


38  percent  as  much  for  the  simulator  for 
all  costs  over  a  15-year  period.  The  simu¬ 
lator  cost  less  to  procure  ($595  thousand 
vs  $2105  thousand,  or  28  percent  as  much) 
and  less  to  operate  ($1588  thousand  vs 
$3367  thousand  or  47  percent  as  much)  over 
a  15-year  period. 

Therefore,  maintenance  simulators  are 
more  cost-effective  than  actual  equipment 
trainers. 

DISCUSSION 

The  finding  that  maintenance  simula¬ 
tors  are  more  cost-effective  than  actual 
equipment  trainers  is  necessarily  qualified 
by  the  limited  nature  of  the  data  from 
which  it  is  derived.  Effectiveness,  as 
used  here,  is  based  on  performance  demon¬ 
strated  at  school  rather  than  on  the  job. 
Cost,  as  used  here,  refers  to  the  initial 
costs  of  acquiring  training  equipment  and 
does  not  include  the  costs  associated  with 
the  long  term  use  of  simulators  or  of 
actual  equipment  for  training,  e.g.,  main¬ 
tenance  and  upkeep,  instructors  and  support 
personnel,  student  pay  and  support.  In  the 
one  case  where  a  life-cycle  cost  comparison 
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(Thousands 

of  dollars) 

Item 

Simulator 

Acquisition 

2105 

595 

Recurring  costs 

3367 

1588 

Total 

5472 

2183 

Net  present  value 
(1978  dollars) 

3896 

1501 

Cost  per  student 
j  hour 

60 

23 

FIGURE  12.  15-Year  Life-Cycle  Costs  of  6883  Test  Stand 
3-Dlmenslonal  Simulator  and  Actual  Equipment 
Trai ner 


was  made,  total  cost  per  student  hour  over 
a  15-year  period  for  the  6883  Test  Stand 
3- Dimensional  Si.nu1  stor  was  38  percent  as 
much  as  for  the  actual  equipment  trainer. 
Both  were  equally  effective  as  measured  by 
tests  at  school  and  by  supervisors'  ratings 
of  performance  of  technicians  on  the  job 
after  leaving  school. 

The  data  on  the  cost  and  effective¬ 
ness  of  maintenance  simulators  have  not 
been  collected  in  a  systematic  manner. 
Therefore,  thera  is  no  basis  at  present 
for  making  trade-offs  between  the  effec¬ 
tiveness  and  cost  of  different  types  of 
maintenance  simulators  on  such  issues  as 
two-dimensional  vs  three-dimensional 
design,  the  complexity  of  maintenance 
simulators  (in  such  terms  as  number  of 
malfunctions  and  instructional  procedures), 
the  extent  to  which  simulators  should 
provide  a  mixture  of  training  in  general 
maintenance  procedures  applicable  to  a 
number  of  different  equipments  or  for  main¬ 
taining  only  specific  equipments,  and  the 
optimum  combination  of  maintenance  simu¬ 
lators  and  actual  equipment  trainers  for 
training  technicians  at  school. 

There  have  been  too  few  studies 
on  the  amount  of  student  time  saved  with 
the  use  of  maintenance  simulators.  There 
have  been  no  studies  on  whether  the  use 
of  maintenance  simulators  influences 
the  amount  of  student  attrition  at  school. 
There  have  been  no  studies  to  collect 
objective  measures  of  performance  of 
maintenance  technicians  on  the  job  after 
training  either  with  simulators  or  actual 
equipment  trainers. 

Maintenance  simulators  now  under 
development  ire  only  beginning  to  use 
recent  technological  advances  such  as 
videodiscs,  automated  voice  input  and 
output,  and  miniaturization  sufficient  to 
make  them  readily  portable.  There  has  been 
more  talk  than  action  about  such  poesibi,’  i- 


ties.  Reductions  in  size  would  make  it 
possible,  as  well  as  convenient,  to  use 
maintenance  simulators  for  refresher  train¬ 
ing  near  job  sites  and  for  performance 
evaluation  and/or  certification  of  mainte¬ 
nance  personnel  on  an  objective  basis  in 
operational  environments.  Extreme  reduc¬ 
tions  in  size  would  make  it  possible  to  use 
maintenance  simulators  as  job  aids  in  per¬ 
forming  maintenance  on  operational  equip¬ 
ment,  thus  assuring  a  close  link,  not  yet 
available,  between  facilities  used  for 
training  at  school  and  for  performance  on 
the  job.  There  is  a  small  but  probably 
insufficient  effort  along  these  lines. 

CONCLUSIONS 

1.  Maintenance  simulators  are  as 
effective  as  actual  equipment  trainers  for 
training  military  personnel,  as  measured 
by  student  achievement  at  school  and,  in 
one  case,  on  the  job.  The  use  of  mainte¬ 
nance  simulators  saves  some  of  the  time 
needed  by  students  to  complete  courses, 
but  data  on  this  point  are  limited.  Stu¬ 
dents  favor  the  use  of  maintenance  simu¬ 
lators;  instructors  are  favorable,  neutial 
or  negative  to  the  use  of  simulators  in 
about  equal  amounts. 

2.  The  acquisition  cost  of  mainte¬ 
nance  simulators  varies  from  20  to  60  per¬ 
cent  that  of  actual  equipment  trainers,  for 
cases  where  complete  cost  data  were  avail¬ 
able.  The  higher  value  includes  the  costs 
of  research  and  development  needed  to  pro¬ 
duce  one  unit;  the  lower  value  includes 
only  unit  recurring  fabrication  costs,  one 
life-cycle  cost  estimate  shows  that  pur¬ 
chase  and  use  of  a  simulator  would  cost  38 
percent  as  much  over  a  15-year  period  as  it 
would  for  an  actual  equipment  trainer. 

3.  Maintenance  simulators  are  as 
etfective  as  actual  equipment  trainers  for 
training  maintenance  personnel.  They  cost 
less  to  acquire.  Therefore,  maintenance 
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simulators  are  cost-effective  compared  to 
actual  equipment  trainers. 

4.  The  conclusions  to  this  paper  must 
be  qualified  by  the  fact  that  they  are 
based  on  limited  and  often  incomplete  data. 
There  is  a  need  for  hard  data  that  compare 
maintenance  simulators  to  actual  equipment 
trainers  in  the  following  areas:  life-cycle 
costs,  on-the-job  performance,  and  student 
attrition  at  school.  There  is  also  a  need 
to  compare  the  cost  and  effectiveness  of 
simulators  that  vary  in  complexity  of 
design,  e.g.,  two-  and  three-dimensional 
simulators  and  types  of  instructional 
features. 
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ABSTRACT 

Competitive  prototyping  during  full-scale  development  is  an  innovative  concept  and  has  been 
exercised  in  various  Army  programs  such  as  the  Division  Air  Defense  System  (DIVADS)  and  Conduct  of  Fire 
Trainers  (COFT)  for  tanks  and  fighting  vehicle  systems.  This  technique  is  expected  to  be  particularly 
effective  when  combined  with  hands-off,  “skunk  works"  type  management  by  the  Government.  If  success  is, 
in  fact,  achieved,  this  concept  should  be  considered  for  use  in  other  projects  involving  large  capital 
investments. 


TRAINER  DESCRIPTION 

The  Conduct  of  Fire  Trainer  (COFT)  (Figure  1) 
is  a  deployable,  shelterized  gunnery  simulator 
which  uses  computer-based  visual  simulation  tech¬ 
nology  to  create  an  environment  that  is  ideal  for 
learning.  The  COFT  produces  full-color  computer¬ 
generated  action  scenes  (Figure  2)  in  which  tank 
crew  members  can  see  and  interact  with  dynamic 
multiple  target  situations.  Yet,  there  is  no 
danger  to  the  crew,  no  fuel  is  consumed,  and  no 
ammunition  is  expended.  In  addition  to  saving 
fuel  and  ammunition  costs ,  the  COFT  has  other  ad¬ 
vantages:  it  can  be  used  24  hours  per  day,  every 
day;  weather  can  be  scheduled  in  the  simulator: 
simulated  engagements  can  be  reenacted,  and  degra¬ 
ded  modes  of  operation  can  be  practiced.  The  COFT 
allows  more  tanks  to  be  kept  where  they  should  be: 
ready  for  any  emergency. 

Training  Capabilities 

By  simulating  a  wide  variety  of  situations 
and  tactical  engagements,  the  COFT  system  can 
provide  basic  gunnery  training  and  keep  fully 
qualified  crews  proficient.  A  library  of  pre¬ 
programmed  exercises  is  provided  which  can  be 
loaded  and  executed  from  the  instructor  station. 

A  training  sequence  should,  typically,  progress 
from  identifying  a  target  to  setting  up  the 
weapons  system  to  aiming  the  reticle  and  firing 
the  simulated  weapon.  The  COFT  system  will  simu¬ 
late  different  times  of  day  or  night,  including 
dawn  and  dusk.  Simulated  special  effects  include 
rain,  smoke,  variable  fog,  and  fading  to  further 
increase  scene  realism. 

BACKGROUND 

Published  material  dealing  with  DOD  philo¬ 
sophy,  past  experience  and  lessons  learned  with 
competitive  prototyping  during  full-scale  develop¬ 
ment  using  the  "skunk  works"  philosophy  is  vir¬ 
tually  non-existent.  This  paper  may,  in  fact,  be 
the  first  formal  document  which  addresses  this 
subject  and  could  be  used  as  a  baseline  for  suc¬ 
ceeding  studies. 

In  September,  1979,  compecing  contracts  were 
awarded  by  the  Naval  Training  Equipment  Center 
for  research  and  development  of  a  Conduct  of  Fire 
Trainer  as  described  above  for  the  Ml  and  M60 
tanks  and  subsequently  the  Fighting  Vehicle  Sys¬ 
tems  (FVS) .  Following  delivery  of  the  trainer,  a 
competitive  test  of  the  trainers  and  evaluation 
of  production  proposals  was  scheduled  with  the 
winner  to  be  awarded  a  multi-year,  multi-million 


dollar  production  contract.  The  RSD  contract 
awarded  to  General  Electric  and  to  Chrysler  Defense, 
Inc.  followed  very  closely  an  earlier  "skunk  works" 
type  contract  which  the  Army  had  executed  for  the 
DIVAD  gun. 

Competitive  prototyping  is,  of  course,  not  new 
to  the  acquisition  process.  However,  it  has  his¬ 
torically  taken  place  dpring  validation  with  award 
of  a  full-scale  development  contract  to  the  winning 
contractor  followed  by  delivery  of  a  complete  Tech 
Data  Package,  design  disclosure  and  a  package  for 
solicitation  of  a  production  proposal,  as  was  the 
the  case  with  the  Ml  Tank. 

"Skunk  works"  contracting  also  is  not  a  new 
initiative  in  acquisition  since  it  was  used  as 
early  as  1943  by  the  Air  Force  on  a  contract  with 
Lockheed  Aircraft  Corporation  for  the  P80*,  the 
first  tactical  jet  fighter  aircraft.  It  was  again 
used  for  contracting  for  the  U2  spy  plane  in  1955, 
and  on  the  SR71  Reconnaissance  Aircraft  between 
1960  and  1975.  It  has  been  used  a  total  of 
eighteen  times  at  Lockheed  (sole  source)  and  the 
procedure  is  considered  by  Lockheed  to  be  very 
successful.  The  refinements  that  have  been  applied 
to  the  present  efforts  are: 

1.  The  competitive  prototyping  is  taking 
place  during  full-scale  engineering  development 
and  is  followed  by  a  competitive  test  resulting  in 
production  award  to  the  winning  contractor  within 
three  years  of  the  development  contract  award; 

2.  The  "skunk  works"  is  taking  place  during 
a  competitive  phase  as  opposed  to  the  sole  source 
environment. 

For  clarity,  the  philosophy  above  will  be 
referred  to  as  "skunk  works  81"  in  the  remainder  of 
this  paper. 

SKUNK  WORKS  81 

What  Is  It? 

It  is  the  technique,  as  explained  above,  of 
utilizing  competition  and  minimum  interaction  with 
the  Government  in  achieving  the  best  of  both  worlds 

*  Information  was  collected  via  phone  conversation 
with  Mr.  "Kelly"  Johnson,  former  director  of  the 
Advanced  Development  Products  Group  at  Lockheed 
Aircraft  Corporation  and  original  executor  of  the 
"skunk  works"  philosophy. 
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by  optimizing  technical  development  at  minimum 
cost. 

The  "skunk  works"  designation  implies  indus¬ 
try's  achievement  of  designated  goals  with  little- 
or  no  Government  control.  In  the  case  of  training 
devices,  contractors  have  been  given  the  goals  of 
training  effectiveness  equal  to  or  exceeding  that 
of  training  on  the  weapon  system  and  minimum  life- 
cycle  cost.  Much  has  been  said  in  the  past  about 
the  expensive,  overly  restrictive  requirements  of 
Government  specifications.  With  the  "skunk  works 
81"  training  device  philosophy,  industry  m  ist 
decide  where  and  in  what  ways  the  requirements  are 
overly  restrictive  in  achieving  and  maintaining 
the  goal  of  training  effectiveness  and  afford¬ 
ability  (low  life-cycle  cost)  and  make  appropriate 
trade-offs. 

When  Is  It  Used? 

The  prerequisite  is  a  long  production  base 
and  a  procurement  budget  large  enough  to  amortize 
the  increased  RSD  dollars  involved  in  awarding  two 
development  contracts.  In  the  case  of  COFT,  the 
production  base  was  recognized  to  be  at  least 
several  hundred  million  dollars  programmed  for 
four  years  or  longer. 

How  Is  It  Used? 

Award  Criteria  -  It  must  be  made  in  a  fixed 
price  environment.  In  the  case  of  COFT,  it  is 
cost  plus  with  a  ceiling  which,  in  effect,  achieves 
the  same  objective  as  fixed  price.  In  any  other 
type  environment,  the  development  period  could 
easily  become  a  spending  contest  between  the  com¬ 
peting  contractors. 

Monitoring  -  Generally,  in-plant  monitoring 
is  limited  to  that  required  to  keep  the  Government 
current  as  to  each  contractor's  progress.  The 
Government  reserves  the  right  of  termination  in 
the  event  that  progress  is  not  satisfactory. 
"Splinter  Groups"  of  Government  and  contractor 
personnel  are  used  at  Progress  Reviews  in  order  to 
facilitate  effective  communication  and  provide  a 
sufficiently  detailed  and  accurate  assessment  of 
progress. 

Technical  Requirements  -  Intentionally  very 
loose,  and  generally  used  as  guidelines  only.  The 
Government  is  buying  a  contractor's  "best  effort" 
and  insists  on  only  certain  basic  requirements 
(e.g.,  safety,  production  readiness)  and  baseline 
documentation  necessary  to  conduct  production  pro¬ 
posal  evaluation  and  device  testing.  The  produc¬ 
tion  contract  will  be  awarded  to  the  contractor 
with  an  acceptable  product ■ that  is  the  most  train¬ 
ing  and  cost  effective. 

Acceptance  Criteria  -  The  only  criteria 
necessary  for  the  acceptance  of  the  RSD  model  by 
the  Government  is  count,  condition  and  safety. 

Other  factors,  however,  must  be  assessed  before 
beginning  OT. 

Logistics  -  Obviously,  a  complete  logistics 
package  on  both  designs  would  be  too  costly. 
Therefore,  a  modified  LSAR  has  been  incorporated 
which  attempts  to  achieve  the  best  of  both 
worlds;  that  is,  delaying  the  expensive  portion 
of  the  process  until  after  selection  of  one  design 
but  assuring  that  either  contractor  is  ready  to 
furnish  logistics  data  in  a  timely  manner  when 
needed. 

Security  -  Communication  is  limited  to  that 
necessary  to  satisfactorily  assess  progress. 


Statements  of  non-disclosure  have  been  made  by  all 
Government  participants. 

Contract  Data  Requirements  List  (CDRL) 

Only  those  data  items  necessary  to  assure 
that  the  Government  will  have  appropriate  informa¬ 
tion  when  necessary  are  required.  For  instance, 
the  Configuration  Item  Development  specification 
(CIDS)  is  required  and  will  be  used  as  a  baseline 
for  the  contractors'  initial  production  proposal. 
Therefore,  this  item  must  be  delivered  in  accor¬ 
dance  with  the  specified  requirement.  Likewise, 
the  Government  acceptance  test  procedure  will  be 
used  to  verify  that  the  CIDS  is  an  accurate  repre¬ 
sentation  of  the  prototype  being  tested,  so  it 
must  also  be  delivered  in  accordance  with  the 
Government  requirement. 

Test  Planning 

An  unusual  characteristic  is  that  operation¬ 
al  test  planning  must  be  done  in  accordance  with 
the  goals  that  were  given  the  contractor  by  the 
developer  at  the  onset  of  the  development  period, 
i.e.,  if  training  effectiveness  was  the  most 
important  goal  then  it  must  assume  the  most  impor¬ 
tant  aspect  of  the  test.  However,  the  role  of  the 
operational  tester  is  supposed  to  be  independent 
of  the  developer;  therefore,  the  developer  must 
input  heavily  to  the  structure  of  the  operational 
test  so  that  the  goals  will  be  adequately  tested. 

What  Are  the  Pros  and  Cons? 

Industry  -  Advantages:  1)  Contractors  are 
given  trade-off  authority  within  the  specifica¬ 
tions  and  can  follow  their  own  interpretation  in 
achieving  the  best  mix  of  training  effectiveness 
and  affordability;  2)  There  is  an  absolute  minimum 
of  Government  interference;  3)  Enthusiasm  on  the 
industrial  project  team.  Unique  opportunities  to 
"do  their  own  thing"  encourages  intitative;  4)  No 
extensive  design  reviews,  per  se. 

Industry  -  Disadvantages:  1)  Without  Govern¬ 
ment  specifications,  can  RAM  be  achieved  to  pro¬ 
vide  low  life-cycle  cost?  (In  some  cases,  author 
understands  that  Government  specifications  were 
voluntarily  used) ;  2)  Competition  could  force  con¬ 
tractors  to  invest  company  funds;  3)  The  idea 
looks  good  but  can  you  really  keep  "big  brother" 
out?  4)  In  many  cases,  one  individual  is  watching 
progress  of  both  contractors.  Thus,  proprietary 
design/security  must  be  a  consideration. 

Government-Advantages :  1)  Competition 
achieves  end  result  (better  than  aggressive  con¬ 
trol  of  a  single  source) ;  2)  Responsibility  for 
the  design  not  placed  on  the  Government;  3)Fewer 
data  items  to  monitor,  less  paper  work  (however, 
this  may  be  offset  by  having  data  items  delivered 
concurrently  by  two  contractors,  thus  causing 
heavy  involvement  by  the  reviewing  Government  team 
members  at  possibly  critical  times) f  4)  Fewer 
Requests  for  Deviation  (RFD) ,  Requests  for  Waiver 
(RFW)  ,  and  Engineering  Change  Proposals  (ECP)  to 
process;  5)  Competition  drives  down  cost,  and  at 
the  same  time  drives  technical  capability  up. 

Government  Disadvantages:  1)  Very  large 
source  selection  team  (two  contractors,  two  pro¬ 
duction  proposals,  two  sets  of  test  reports  to 
review,  etc);  2)  Less  directionary  authority  but 
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more  effort  in  keeping  current  on  two  designs;  less 
opportunity  to  gain  insight  into  the  design;  3) 
same  security  problem  as  industry  concern  above; 

4)  Greater  than  average  risk  of  protest  from  losing 
contractor;  5)  No  control  over  design,  no  know¬ 
ledge  of  costs,  manhours,  etc  to  apply  to  produc¬ 
tion  proposal  evaluation. 

SPECIAL  OPERATING  PROBLEMS 

Monitoring 

There  is  a  tendency  for  upper  management  to 
rely  completely  on  the  theoretical  skunk  works 
(hands-off)  philosophy.  Given  today's  competition 
for  people  resources  within  most  Government  agen¬ 
cies,  various  managers  may  have  a  tendency  to  view 
this  approach  as  an  opportunity  to  withdraw  the 
support  necessary  for  detailed  attention  during 
the  RfiD  phase.  However,  it  is  in  the  best  public 
interest  for  the  Government  to  assure  that  all 
steps  are  taken  to  maintain  competition.  There¬ 
fore,  appropriate  conmmnications  between  the  Gov¬ 
ernment  and  the  contractors  must  be  maintained 
while  respecting  the  competitive  environment. 

Resources 

All  Government  costs  increase.  Supporting 
manhours  roughly  double  with  the  necessity  to 
monitor  two  designs. 

Data 


CONCLUSIONS 

"Skunk  works  81"  has  proven  to  be  a  viable 
method  of  developing  training  devices.  Initial 
indications  are  that  competition  has  favorably 
influenced  both  technical  development  and  cost. 

However,  all  the  returns  are  not  yet  in. 

We  have  identified,  at  the  upper-management  level, 
the  major  problems  encountered  and  expected  in 
managing  a  "skunk  works  81"  acquisition.  To  make 
the  maximum  use  of  lessons  learned,  panels  should 
be  developed  in  order  to  investigate  the  problems 
more  thoroughly  using  logisticians,  contracting 
officers,  testers,  RAM  and  design  engineers  and 
managers  who  have  experience  in  this  type  of 
acquisition  and  properly  record  these  lessons 
learned  for  future  use. 
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Certain  data  items  cannot  be  considered 
under  the  blanket  of  "best  effort. "  As  discussed 
earlier,  contract  data  requirements  must  be  form¬ 
ally  assembled,  delivered  and  accepted  within  the 
terms  of  the  contract. 

Logistics  Planning 

This  will  be/has  been  extremely  difficult  in 
light  of  the  fact  that  the  majority  of  the  data 
available  will  not  be  delivered  until  after  source 
selection.  This  is  not  timely  nor  is  it  in  accor¬ 
dance  with  AR-700-127.  Many  problems  are  posed  as 
a  result  of  the  lack  of  design  information  avail¬ 
able  early  in  development.  For  instance,  facili¬ 
ties  planning  .technical  manuals  and  provisioning 
all  start  later  in  the  development/production 
cycle  than  the  Army  logistics  system  demands  it. 
This  will  probably  drive  ownership  costs  up  in  the 
early  years  until  the  wholesale  logistics  system 
can  fully  support  the  equipment. 

Source  Selection 


Competition  forces  detailed  assessment  of 
the  Operational  Test  by  the  Source  Selection 
Board.  The  question  is  not  only  the  traditional, 
"Will  the  equipment  be  accepted  into  the  inven¬ 
tory"?  but  adds  a  new  dimension,  "Which  one"? 
Because  of  this  increased  effort,  lack  of  depth 
in  various  talent  areas  forces  the  Project  Manager 
to  seek  assistance  from  other  Government  offices. 
This,  in  turn,  forces  expenditure  of  TDY  funds  and 
funds  to  expand  facilities,  since  the  evaluation 
should  take  place  in  one  area  as  a  security  pre¬ 
caution.  It  is  important  to  identify  these 
resources  early. 
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IMPROVING  PROGRAM  START-UPS 


Allic  L.  Waldron 
Program  Manager 
AAI  Corporation 
Cockeysville,  Maryland  21030 


The  current  modus  operandi  for  the  Initiation  of  large  simulator  programs  Is  "throttles 
to  the  firewall"  on  the  day  of  contract  award.  The  contractor's  program  management  team 
doesn't  have  time  to  effectively  plan  the  program  due  to  an  Immediate  and  continuous  barrage 
of  customer  visibility  data  Items  and  reviews.  This  Inability  to  develop  accurate  and 
workable  plans  Impacts  program  costs  end  schedules  by  means  of  false  starts  and  premature 
commitment  of  resources.  Likewise  the  government  program  office  incurs  unneeded  travel 
and  workload  commitments  during  the  initial  stages  of  their  organizing  and  planning  effort. 
This  problem  is  caused  by  an  attempt  to  achieve  the  earliest  possible  Ready  for  Training 
date  through  rapid  start-up  and  close  monitoring  of  progress.  The  solution  lies  in  a  joint 
Government /Contractor  front  end  planning  effort  which  allows  the  contractor  time  to  plan 
and  also  provides  the  government  with  sufficient  visibility  during  the  start-up  period. 


Over  the  past  few  years  a  common  approach 
for  the  start-up  of  large  simulator  programs  has 
developed  which  is  in  essence  "throttles  to  the 
firewall"  on  the  day  of  contract  award.  Current 
RFP  requirements  clutter  the  critical  early 
months  of  these  programs  with  a  continuous 
barrage  of  premature,  and  in  some  cases 
unnecessary,  reviews  and  data  item  submittals. 

The  shear  magnitude  of  these  "square-filling" 
requirements  has  a  detrimental  effect  on  the 
program  management  team's  ability  to  plan 
effectively.  This  inability  to  develop 
comprehensive  and  workable  plans  has  a  definite 
impact  on  program  costs  and  schedules  since 
Inadequate  planning  leads  to  false  starts  and 
premature  commitment  of  resources.  Also,  the 
premature  submittal  of  data  and  the  conduct  of 
poorly  timed  reviews  detract  from  the  value  of 
these  requirements,  which  are  most  effective 
when  properly  time  phased. 

The  existence  of  this  problem  presents  both 
a  challenge  and  an  opportunity  to  government  and 
industry  managers.  The  challenge  is  to  develop 
an  approach  to  program  start-up  which  optimizes 
the  needs  of  both  industry  and  government.  The 
opportunity  is  to  reap  the  benefits  to  be  gained 
from  comprehensive  front  end  planning  and  proper 
time  phasing  of  requirements. 

The  scope  of  the  problem  at  hand  can  best 
be  illustrated  via  a  current  example.  An  RFP 
for  an  operational  flight  trainer  released 
within  the  last  few  months  requires  the  following 
data  items  to  be  submitted  within  the  first  120 
days  after  contract  award, 

-  Operation  and  Support  Cost  Trade  Studies 

-  Firmware  Development  Plan 

-  Computer  Program  Development  Plan 

-  Program  Milestones 

-  Contract  Work  Breakdown  Structure 

-  Monthly  Activity  Reports 

-  Support  Equipment  Plan 

-  Conf igurat J.on  Management  Plan 

-  Parts  Control  Program  Plan 

-  Cost  Performance  Report 

-  Training  and  Training  Equipment  Plan 

-  Integrated  Support  Plan 

-  Maintainability  Plan 

-  Reliability  Program  Plan 

-  Facility  Design  Criteria 

-  Support  Equipment  Recommendations  Data 

-  Systems  Engineering  Management  Plan 


-  Logistics  Support  Analysis  Record  Data 

-  Logistics  Support  Analysis  Plan 

In  addition,  the  program  management  team  must 
schedule,  prepare  for,  conduct  and  submit  minutes 
for  the  following  reviews  during  this  same  120 
day  period. 

-  Post  Award  Conference 

-  System  Requirements  Review 

-  Engineering  Design  Review 

-  Computer  Program  System  Documentation 
Guidance  Conference 

-  2  Program  Management  Reviews 

-  Project  Planning  Review 

Obviously,  these  requirements  constitute  a 
substantial  workload  for  the  Program  Manager  and 
his  staff  which  significantly  impacts  their 
priorities  and  effectiveness  during  a  very 
critical  phase  of  the  program. 

Admittedly,  these  data  items  and  reviews  are 
germain  to  overall  program  planning  and  control 
and  provide  the  customer  with  needed  visibility. 
However,  they  are  not  all  needed  at  this  early 
stage  of  the  program. 

There  is  no  single  cause  on  which  we  can 
place  the  blame  for  this  problem.  However,  the 
main  driving  factor  influencing  government  RFP 
writers  is  the  very  real  need  to  achieve  the 
earliest  possible  RFT  and  begin  realizing  the 
very  tangible  cost  savings  provided  by  modern 
trainers.  The  RFP  writers  have  assumed,  quite 
correctly,  that  one  of  the  best  ways  to  insure 
program  success  is  to  plan  well  and  maintain 
visibility  as  to  progress  against  that  plan. 
Unfortunately,  the  schedules  for  completing 
planning  requirements  in  current  RFP's  have  been 
compressed  by  a  desire  to  demonstrate  immediate 
results  and  progress  to  the  chain  of  command. 

Also,  the  RFP  requirements  have  grown  more 
extensive  and  complicated  due  to  the  influence  of 
popular  paneceas  or  DOD  crusades  such  as  getting 
control  of  software  by  planning,  documenting,  and 
reviewing  it  to  death.  In  addition,  there  is  now 
and  always  has  been  the  problem  of  the  use  of 
strawmen  in  writing  RFP's  and  treating  the 
Acquisition  Management  Systems  and  Data  Requirement 
Control  List  (DOD  5000. 19-L)  as  a  data  item 
shopping  list.  As  a  result,  each  RFP  or  CDRL  has 
all  the  requirements  specified  in  the  last  RFP 
plus  any  new  ones  which  have  been  directed  from 
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above  or  created  from  within  the  procuring 
agency. 

■  Finally,  industry  haa  a  share  in  the  blame 
for  tho  current  start-up  procedures.  In  some 
cases,  contractors  have  been  known  to  delay 
putting  sufficient  resources  on  a  low  visibility 
program  in  order  to  show  progress  on  a  high 
visibility  effort.  Hence,  the  government  manager 
believes  that  his  only  recourse  is  to  insure  that 
his  contract  requires  data  and  reviews  sufficient 
to  keep  the  pressure  on  and  reveal  any  lack  of 
contractor  emphasis  before  his  program  is 
affected. 

The  impact  of  all  these  reviews  and  data 
submittals  on  the  industry  is  dramatic.  For 
instance  in  the  example  cited  above  there  are 
seven  reviews  required  during  the  first  120  days 
of  the  program.  According  to  the  Statement  of 
Work  contained  in  that  RFP,  each  of  these  will 
last  at  least  three  days.  Allowing  two  days  for 
internal  dry  runs,  two  days  for  preparing 
presentations  and  one  day  for  preparing  the 
minutes  of  each  of  their  reviews,  we  see  that  a 
total  of  56  days  could  be  consumed  in  satisfying 
these  requirements.  That  accounts  for  over  46X 
of  the  first  120  days.  Add  to  this  the  time 
required  for  the  data  items  and  it  becomes 
obvious  that  the  program  management  cadre  will 
not  be  able  to  effectively  plan  their  manpower 
requirements  or  develop  comprehensive  schedules 
and  time  phased  budgets  during  this  same  period. 

What  usually  happens  is  that  a  lot  of 
additional  manpower  is  applied  to  the  program 
much  earlier  than  the  real  job  of  building  a 
trainer  requires.  This  additional  manpower 
cranks  out  vugraphs  and  creates  plans  which  are, 
to  a  large  extent,  eyewash  and  boilerplate. 

This  procedure  often  generates  unrealistic  plans 
and  schedules  which  cause  more  work  later  when 
they  must  be  revised  and  the  revisions  justified. 
The  real  danger,  however,  lies  in  the  potential 
for  false  starts  made  in  an  attempt  to  show  early 
progress.  This  results  in  the  expenditure  of 
resources  which  cannot  be  recovered. 

For  example,  the  requirements  for  a  Computer 
Program  Development  Plan  are  so  extensive  and 
detailed  that  they  cannot  be  totally  satisfied 
until  the  system  is  complete  and  tested.  Such 
details  as  module  and  subroutine  definitions 
and  completion  schedules  are  not  not  available 
during  the  first  few  months.  However,  the 
requirement  for  a  plan  with  detailed  schedules 
exists  and  must  be  satisfied.  In  attempting  to 
define  these  items  prematurely,  the  contractor 
often  makes  assumptions  which  are  sometimes 
totally  wrong.  Nevertheless,  the  die  is  cast  and 
the  government  has  a  plan  against  which  to 
monitor  progress.  In  order  to  show  progress, 
the  contractor  starts  to  work  on  the  assumed 
module/subroutine  structures.  If,  as  usually 
happens  during  development,  the  assumptions 
prove  wrong,  significant  resources  have  been 
wasted. 

On  the  government's  side,  the  impact  is 
equally  dramatic.  The  56  days  of  formal  reviews 
normally  entail  56  days  of  travel  time  for 
large  numbers  of  government  employees.  This 
imposes  a  substantial  drain  on  available  travel 
funds  early  in  the  program  and  usually  forces 


cutbacks  in  travel  later  on  when  the  benefits  would 
perhaps  be  greater.  Also,  the  mass  of  data  being 
generated  and  delivered  must  be  reviewed  and  in 
most  cases  approved  by  the  system  program  office 
staff  which  is  being  reorganized  and  expanded  to 
administer  the  contract.  Both  of  these 
requirements  place  a  tremendous  workload  on 
personnel  who  are  just  coming  on  board  and  need 
time  to  acclimate  to  new  jobs  and  responsibilities. 

For  example,  another  recent  simulator  program 
required  the  delivery  of  extensive  documentation 
30  days  prior  to  PDR  which  was  scheduled  to  last 
for  one  week.  The  documentation  was  delivered  as 
scheduled  and  consisted  of  approximately  7,000 
pages  of  technical  data.  The  government  then 
had  thirty  days  to  review  that  data  and  five  days 
to  discuss  it  with  the  contractor. 

Finally,  the  strawman/shopping  list  approach 
to  preparing  RFP's  often  leads  to  the  purchase 
of  expensive  documents  which  serve  little  useful 
purpose  in  the  life  of  the  program.  This  fact 
was  recognized  and  addressed  by  the  AF  in  the 
early  70's.  At  that  time  the  then  Brigadier 
General,  Alton  B.  Slay,  headed  up  a  Data  Reduction 
Action  Group  (DRAG)  which  reviewed  the  data 
requirements  of  major  Air  Force  procurements. 

This  review  indicated  that  large  portions  of  the 
data  being  procured  was  not  in  any  way  being  used 
to  manage  the  programs.  As  a  result  substantial 
reductions  in  data  requirements  were  made  to 
subsequent  AF  RFP's.  However,  it  now  appears 
that  the  DRAG  has  faded  into  history  and  data 
procurement  is  again  getting  out  of  hand. 

From  the  government's  viewpoint  the  solution 
to  the  problem  would  be  for  the  contractor  to  have 
done  such  an  excellent  Job  in  preparing  his 
proposal  that  all  plans  and  schedules  are  ready 
to  go  immediately  after  contract  award.  This  is 
not  possible  because  the  time  allocated  for 
proposal  preparation  and  the  page  limitations 
imposed  effectively  limit  the  detail  which  can 
be  provided.  Even  if  such  accurate  and  detailed 
plans  were  provided  in  the  proposal,  they  would 
be  largely  negated  by  the  fact  that  the  actual 
contract  award  is  in  most  cases  much  later  than 
that  projected  in  the  RFP.  This  lag  allows 
conditions  to  change  relative  to  the  amount  and 
type  of  resources  available  to  the  program. 

From  the  contractor's  viewpoint  the  solution 
would  be  for  the  government  to  award  the  contract 
then  go  away  for  about  six  months  while  top  level 
system  design  and  plans  for  completing  the  program 
can  be  formulated.  At  the  end  of  this  period  the 
government  would  be  provided  with  a  program  plan 
and  schedule.  Clearly,  this  approach  is  not 
possible  because  of  the  large  sums  of  money  being 
committed  and  the  importance  of  the  programs  to 
the  national  defense. 

Therefore,  the  optimum  solution  must  lie 
somewhere  between  these  two  extremes.  It  must 
allow  the  RFP  to  require  and  obtain  pricing  for 
the  reviews  and  data  that  the  procuring  activity 
needs  while  at  the  same  time  providing  the 
contractor  the  time  and  flexibility  to  plan 
properly.  To  accomplish  this  objective  the  RFP 
should  specify  the  reviews  and  data  which,  after 
careful  consideration  and  appropriate  justifica¬ 
tion,  will  be  required  to  manage  the  particular 
program  at  hand.  Instead  of  specifying  data 
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delivery  or  Program  review  dates,  the  contractor 
should  be  tasked  to  provide  his  front  end  plan 
for  the  program.  This  plan  should  be  complete 
not  later  than  90  days  after  contract  award  and 
should  provide  schedules  for  management  and 
design  reviews.  It  should  also  contain  the 
contract  work  breakdown  structure  (CWBS)  and 
specify  when  schedules  for  each  element  of  the 
CWBS  will  be  completed. 

This  90  day  period  is  a  critical  period  for 
the  government  management  team  during  which  they 
must  make  sure  the  program  is  getting  off  the 
ground.  Therefore,  they  should  participate  in 
the  front  end  planning  process  to  provide 
required  visibility  and  to  help  develop  an 
acceptable  plan.  This  participation  should  be 
limited  to  a  few  key  people  and  consist  of  a 
"work  with"  rather  than  a  "watch  over"  effort. 

The  government  project  engineer,  a  software 
expert,  someone  from  the  program  control  function 
and  someone  with  extensive  operational  experience 
with  the  system  being  simulated  would  be  ideal. 

As  part  of  this  effort,  delivery  dates  for 
all  required  data  should  be  time  phased  with  key 
program  events  such  as  PDR  and  CDR,  and  the 
degree  of  completeness  of  the  data  appropriate 
to  those  events  should  be  clearly  specified. 

Again  using  the  Computer  Program  Development  Plan 
as  ai>  example,  the  portions  and  level  detail  to 
be  submitted  at  PDR  and  CDR  should  be  estab¬ 
lished. 

Finally,  the  plan  should  show  how  progress 
will  be  measured  against  the  plan  and  how  status 
will  be  reported  to  the  procuring  agency.  For 
Instance,  the  milestones  to  be  used  in  tracking 
computer  program  development  should  be  jointly 
defined  between  the  government  and  the  contractor 
as  well  as  the  criteria  which  must  be  met  before 
the  milestone  is  considered  complete.  This  will 
alleviate  much  confusion  usually  generated  by 
different  definitions  of  milestones  and  what 
their  completion  means. 

The  first  management  review  with  the  full 
government  management  team  should  then  occur 
approximately  120  days  after  contract  award. 

This  review  would  be  a  modification  of  the 
currently  popular  Program  Planning  Review. 
However,  instead  of  presenting  a  re-hash  of  the 
RFP,  the  contractor  will  be  revealing  his 
approach  to  planning  and  control  which  is 
applicable  to  the  program  in  question,  and  is 
based  on  conditions  which  exist  at  that  point 
in  time.  This  review  should  be  conducted  as  a 
working  meeting  with  the  objective  of  reaching 
an  understanding  of  the  program  plan,  defining 
and  agreeing  to  revisions  to  the  plan  needed  to 
meet  government  requirements  and  achieving  a 
mutual  agreement  relative  to  objectives  to  be 
achieved  and  measures  of  accomplishment. 

At  the  conclusion  of  the  Program  Planning 
Review  the  front  end  plan  is  approved  by  the 
procuring  agency  and  used  to  measure  progress 
until  all  elements  of  the  CWBS  are  completely 
planned. 

This  approach  will  provide  the  contractor 
with  enough  time  to  effectively  plan  the  program. 
He  can  use  his  initial  staff  of  experienced 
manpower  to  formulate  the  overall  concepts  for 


the  program  and  establish  realistic  goals  and 
checkpoints.  It  will  enable  him  to  generate  more 
realistic  schedules  by  allowing  him  to  defer  the 
preparation  of  schedules  for  downstream  efforts 
until  he  has  sufficient  data  to  accurately  scope 
these  efforts.  In  turn,  this  will  help  prevent 
false  starts  caused  by  the  initiation  of  efforts 
which  are  not  adequately  defined.  Finally,  it 
will  give  the  contractor's  upper  management  the 
opportunity  to  review  the  Program  Manager's 
concepts  before  significant  resources  are 
committed  to  the  program. 

To  the  government's  benefit,  early  visibility 
will  be  provided  by  the  participation  in  the  front 
end  planning  process,  the  Program  Progress  Review 
and  subsequent  statusing  of  the  front  end  plan. 
Travel  funds  will  be  conserved  until  properly 
time  phased  reviews  have  been  defined.  The 
system  program  office  will  be  given  enough  time 
to  complete  its  organization  and  indoctrination 
of  newly  assigned  personnel.  Also,  the  Improved 
accuracy  of  the  resulting  schedules  for  the  CWBS 
elements  will  increase  the  credibility  of  program 
status  as  measured  against  those  schedules. 

One  final,  and  possibly  the  most  Important, 
benefit  offered  to  both  parties  by  the  proposed 
front  end  planning  effort  is  the  fostering  of  a 
team  spirit  among  the  contractor  and  procuring 
agency  personnel.  The  fact  that  both  parties 
have  participated  in  the  definition  of  objectives 
improves  the  probability  that  they  will  succeed 
in  meeting  those  objectives  and  achieve  the 
earliest  possible  RFT  date. 

In  conclusion,  it  is  apparent  that  improve¬ 
ments  can  be  made  in  our  procedures  for  initiating 
large  simulator  programs  and  that  those  improve¬ 
ments  will  require  the  combined  efforts  of  industry 
and  government  to  achieve.  Perhaps  the  American 
Defense  Preparedness  Association  offers  the  base 
of  talent  from  which  a  joint  interservice/industry 
working  group  could  be  established  to  review  this 
problem  and  recommend  a  course  of  action. 
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ABSTRACT 

In  October  l?80  PM  TRADE  developed  a  Training  Device  Acquisition  Management  Model  (TDAMM) 
which  outlines  the  specific  events  that  must  be  accomplished  in  the  development  and  production  of  training 
devices.  This  paper  will  explain  how  the  DoD  Acquisition  Model  was  tailored  to  accommodate  training  devices 
to  include  the  responsibilities  of  the  Developer,  the  Users  and  the  role  of  major  commands  in  the  acquisition 
cycle.  The  advantages  of  using  the  model  will  be  mentioned  along  with  the  elimination  of  events  which  are  not 
applicable  and/or  are  not  necessary  In  the  development  of  training  devices;  the  benefits  to  the  User  by 
reducing  development  time  and  minimizing  costs.  Moreover,  an  explanation  will  be  included  to  demonstrate 
its  use  in  assisting  the  Project  Directors  in  accomplishing  their  key  events  in  a  methodical  manner,  and 
tailoring  new  developments  to  the  model  with  an  examination  of  the  resultant  acquisition  strategy. 


PM  TRADE  MISSION/CHARTER 

The  Project  Manager  for  Army  Training 
Devices  (PM  TRADE)  is  responsible  to  centrally  direct, 
coordinate  and  support  the  materiel  development  and 
acquisition  activities  of  system  devices  developed  by  PM 
TRADE  as  agreed  upon  by  the  System  Project  Manager, 
non-system  and  non-type  classified  training  devices.  This 
responsibility  encompasses  the  research,  development, 
configuration  management,  product  assurance, 
developmental  tasting,  procurement,  production,  distribu¬ 
tion,  and  integrated  logistic  support  for  the  following 
categories  of  assigned  training  devices: 

1 .  Non-system  trainers 

2.  Non-type  classified 

3.  Synthetic  Flight  Trainer  systems 

4.  Commercially  workable  training  device 

The  PM  TRADE  provides  assistance  to  all 
Army  agencies  in  the  execution  of  their  responsibilities  in 
the  area  of  training  devices.  Particularly,  this  is  the 
filling  of  the  training  device  requirements  of  the  U.S. 
Army  Training  and  Doctrine  Command  (TRADOC)  and 
U.S.  Army  Forces  Command  (FORSCOM). 

Rather  than  focusing  his  attention  on  a  single 
materiel  system  as  do  most  Project  Managers,  the  PM 
TRADE  must  manage  a  multiplicity  of  systems,  each  in 
various  stages  of  the  training  device  life  cycle  from 
concept  formulation  through  production  and  deployment, 
as  applicable. 

COMPLEXITY  OF  OPERATION 

The  operations  of  the  Project  Manager  for 
Army  Training  Devices  are  much  like  the  operations  of 
any  other  Project  Manager.  It  is  the  several  differences 
briefly  discussed  below  that  cause  what  initially  appears 
to  be  a  straight  foreword  project-managed  effort  to 
become  a  highly  complex  management  problem  to  control 
and  integrate  the  efforts  to  manage  the  Training  Device 
Acquisition  process  of  at  least  fifty  individual  training 
device  systems. 


Not  only  must  several  training  device 
developments  be  actively  pursued,  but  they  are  each  in 
different  parts  of  their  acquisition  process.  Some  are  in 
the  early  parts  of  conceptual  development;  or  perhaps  the 
identified  concept  is  being  demonstrated  or  validated. 
Others  are  in  various  stages  of  Engineering  Development 
and  Production.  Finally,  some  may  already  be  deployed 
and  even  nearing  obsolescence.  Each  of  the  "system"  and 
"non-system"  devices,  no  matter  where  it  is  in  the 
acquisition  process,  must  receive  management  attention 
and  be  supported  adequately  to  meet  the  needs  of  the 
user. 

Each  of  the  system  or  non-system  training 
devices  has  its  own  inherent  technical  risks  of 
development  from  simple  or  off-the-shelf  device  to  very 
sophisticated  devices  requiring  advance  technological 
development.  In  between  are  varying  degrees  of 
complexity  from  moderate,  mostly  mechanical  devices  to 
complex  and  software-oriented  devices. 

In  addition  to  the  technology  that  must  be 
called  upon  to  develop  the  devices,  PM  TRADE  has  the 
full  range  of  trainers  to  manage: 

1.  Trainers  for  the  Corps  of  Engineers 
include  such  devices  as  the  Medium 
Girder  Bridge;  highly  complex  items  for 
Air  Defense  Artillery  such  as  the 
ROLAND  Air  Defense  System;  the 
Multiple  Integrated  Laser  Engagement 
Simulation  (MILES)  and  the  Reaction 
Electronic  Equipment  Simulator  (REES) 
for  the  Signal  Corps. 

2.  Armor  trainers  include  the  Conduct-of- 
Fire  Trainers  for  the  Ml  and  M60  series 
Main  Battle  Tanks  and  the  new  Infantry 
and  Cavalry  fighting  vehicles. 

3.  Synthetic  Flight  Trainer  Systems  include 
simulators  for  the  Advanced  Attack 
Helicopters  (AAH)  and  the  COBRA. 
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Finally,  the  PM  TRADE'S  management 
challenges  are  further  complicated  by  not  having  a 
supporting  research  and  development  facility,  a 
supporting  National  Inventory  Control  Point  (NICP)  nor  a 
National  Maintenance  Center  (NMP)  while  at  the  same 
time  having  multiple  customers  and  funding  sources. 

The  number  of  training  devices  being 
developed,  both  system  and  non-system,  the  several 
different  technological  risks  involved,  the  number  of 
categories  of  sevices,  the  number  of  customers  and 
funding  sources  all  act  to  compound  the  very  difficult 
management  problems  thol  are  the  day-to-day 
environment  in  which  the  PM  TRADE  and  his  personnel 
find  themselves.  This  "juggling  act"  required  that  a  more 
efficient  management  system  must  be  found  or  developed 
in  order  to  make  management  resources  available  to  solve 
the  inevitable  critical  problems  that  exist  during  the 
development  of  a  training  device  but  at  the  same  time 
expend  the  exact  the  management  effort  necessary  to 
routinely  guide  the  normal  development  activities. 

An  acquisition  management  model  called  the 
Training  Device  Acquisition  Management  Model  (TDAMM) 
has  been  developed.  This  generic  model  is  specifically 
intended  to  be  applied  to  any  training  device.  The  model, 
however,  follows  DoD  guidance  very  closely.  It  is, 
therefore,  applicable  not  only  to  the  acquisition  of 
training  device  of  all  of  the  military  services,  but  to  the 
acquisition  process  of  any  item  being  developed  and 
acquired  by  any  of  the  military  services.  The  model  is 
then  modified  to  meet  the  unique  characteristics  of  the 
particular  device  being  developed  to  meet  a  requirement. 
At  this  time  the  status  of  each  event  of  the  model  are 
manually  reported.  When  ttie  staff  of  PM  TRADE  gains 
experience  with  the  manual  TDAMM,  the  model  has  been 
designed  with  the  characteristic;;  that  it  can  be  easily 
automated.  This  is  already  in  the  advanced  planning 
stage.  The  automated  version  of  the  TDAMM  will, 


henceforth,  be  immediately  responsive  to  training  device 
managers  telling  them  the  status  of  a  training  device 
development.  It  bears  repeating;  the  TDAMM  can  be 
applied  to  any  training  device  of  any  service  or  any  other 
item  being  developed. 

TRAINING  DEVICES  AND  THE 
ACQUISITION  MANAGEMENT  MODEL 

It  is  the  goal  of  DoD  that  all  acquisitions  of 
military  materiel  follow  the  Life  Cycle  Systems 
Management  Model.  The  p'ocess  for  conceiving, 
developing,  acquiring  and  fielding  new  items  of  materiel 
is  formalized  in  the  Life  Cycle  System  Management 
Model  (LCSMM).  Marxigement  of  the  development  of  a 
single  training  device  (or  all  training  devices  if  each  is 
considered  independently  of  all  other  training  devices)  fits 
the  LCSMM.  This  is  surely  true  at  least  from  a  general 
point  of  view.  Yes,  there  are  differences  se'-h  ns  holding 
In-Process  Reviews  rather  than  an  ASARC  ard/or  DSARC 
(Army  Systems  Acquisition  Review  Council  and/or 
Defense  Systems  Acqui-sition  Review  Council 
respectively).  Either  the  Vnlidation  Phase  or  the  Full 
Scale  Development  Phase  of  training  device  development 
is  normally  severely  curtailed.  Also,  the  Production  and 
Deployment  Phase  is  often  deleted  due  to  the  limited 
quantities  of  training  devices  needed.  The  Full-Scale 
Development  Phase  often  fills  the  toxal  requirement. 
These  differences  mitigate  against  the  strict  following  of 
every  event  of  the  LCSMM  and  require  the  modification 
or  tailoring  of  the  model  in  order  to  meet  the  needs  of  the 
training  device  development  managers.  In  this  way  the 
baseline  of  the  Training  Device  Acquisition  Management 
Model  (TDAMM)  was  established,  i.e.,  follow  the  Life 
Cycle  System  Management  Model  overlayed  or  modified 
with  the  unique  requirements  of  the  development 
requirements  of  training  devices.  This  approach  is  a 
practical  approach  to  any  development  and  acquisition 
activity. 
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The  relationships  of  the  decision  points, 
documentation,  phases,  testing,  program  funds,  etc.,  are 
shown  in  the  diagram  below.  This  generalized 
representation  of  the  acquisition  process  is  the  baseline 
for  modeling  the  training  device  acquisition  cycle  to 
assure  that  the  training  device  model  and  the  more 
familiar  DoD  materiel  acquisition  model  attain  maximum 
compatibility  with  each  other. 

The  significant  differences  between  the 
LCSMM  and  TDAMM  are  summarized  in  the  following 
chart. 


LCSMM 

TDAMM 

Activity  Relates  To  This 

Activity 

ASARC-II/DSARC-II 

VAL  IPR 

ASARC-lll/DSARC-lll 

DEVA  IPR 

MENS 

TENS 

ROC 

TDR 

DESCRIPTION  OF  THE  TDAMM 


The  Training  Device  Acquisition  Management 
Model  (TDAMM)  has  been  designed  to  assist  the  Project 
Manager  for  Army  Training  Devices  (PM  TRADE)  and  his 
subordinate  elements  in  the  management  of  the  training 
device  development  and  procurement  process;  i.e.,  the 
materiel  acquisition  process.  The  TDAMM  diagrams  and 
the  its  accompanying  explanations  has  the  primary 
purpose  of  servicing  as  a  "path  with  a  series  of  sign-posts" 
in  order  to  know  what  to  do  in  traveling  through  the  total 
Training  Device  Life  Cycle. 

The  Training  Device  Acquisition  Management 
Model,  portrays  all  of  the  202  events  to  be  taken,  or  at 
least  considered,  to  acquire  a  training  device  system  from 
the  time  that  a  requirement  is  stated  by  the  user  through 
the  items  obsolescence,  perhaps  twenty  years  later.  The 


four  phases  of  acquisition  of  a  training  device,  os  for  any 
other  materiel  acquired  wilhin  the  purview  of  the  DoD 
Material  Acquisition  Model  are: 

1.  Need  Identification  and  Concept 
Formulation  Phase. 

2.  Demonstration  and  Validation  Phase 
(also  known  as  Advanced  Development 
(AD)). 

3.  Full-Scale  Development  Phase). 

4.  Production  and  Deployment  Phase. 

The  first  phase  of  TDAMM  has  the  dual 
purpose  of  identifying  training  voids  either  presently 
existing  or  that  will  exist  as  new  equipment  is  introduced 
into  the  Army  inventory.  Secondly,  the  purpose  of  this 
phase  is  to  develop  concepts  of  possible  training  devices 
that  will  meet  the  identified  need. 

During  the  conduct  of  training  by  the  US  Army 
Training  and  Doctrine  Command  (TRADOC)  or  the  other 
Major  Commands  (MACOM)  of  the  US  Army,  deficiencies 
in  the  training  program  will  be  found.  During  the 
development  of  major  materiel  items  the  Mission  Element 
Need  Statement  (MENS)  will  first  identify  the  probably 
need  for  training  devices.  Another  way  to  find  a  need  for 
a  training  device  is  during  the  development  of  training 
programs  which  are  developed  in  order  to  teach 
operational  and  logistic  support  activities.  Anyone  may 
find  such  training  voids  that  could  be  filled  by  a  new  or 
improved  training  device.  When  a  training  void  is  found,  a 
Training  Element  Need  Statement  (TENS)  (corresponding 
to  the  MENS),  describes  the  need.  The  TENS  is  prepared 
by  the  Combat  and  Training  Developer  (normally  the 
TRADOC  proponent  school)  in  coordination  with  the  PM 
TRADE. 

Based  on  the  Training  Element  Need 

Statement  (TENS)  the  Combat,  Training,  and  the  Materiel 


Figure  2  -  CONCEPT  PHASE  (PARTIAL) 
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Developers  prepore  o  Concept  Formulation  Package  and 
an  Outline  Program  Management  Plon  (PMP).  The 
Materiel  Developer  develops  technical  and,  in 
coordination  with  User,  logistic  concepts  based  on  such 
things  as  engineering  analyses,  logistic  support  concepts, 
and  the  Reliability,  Availability  and  Maintainability 
(RAM)  goals.  It  is  emphasized  that  the  activities  that  are 
associated  with  concept  formulation  are  repetitive, 
mutually  supporting,  fully  interrelated  between  the 
Combat,  Training  and  Materiel  Developers  and  are 
continually  being  revised  or  updated.  Moreover, 
throughout  the  CONCEPT  PHASE,  efforts  are  undertaken 
simultaneously  on  as  many  alternative  concepts  as 

practical  so  as  to  meet  the  potentially  broader  goals  of 
the  Training  Element  Need  Statement. 

The  repetitive  process  of  concept  formulation 
demands  the  continued  efforts  to  determine  among  other 
things: 

a.  A  Trade-Off  Determination  (TOD). 

b.  A  Trade-Off  Analysis  (TOA). 

c.  The  Best  Technical  Approach  (BTA). 

d.  Agreement  on  the  system  description. 

e.  Agreement  on  the  Bands-of-Performance 
(BOP)  of  the  characteristics  of  the 
system. 

f.  Preparation  of  a  Cost  and  Training 
Effectiveness  Analysis  (CTEA). 

g.  The  Basis  of  Issue  Plan  (BOIP). 

h.  A  Qualitative  and  Quantitative 
Personnel  Requirements  Information 


(QQPRI)  document. 

I.  A  validated  Baseline  Cost  Estimate 
(BCE). 

j.  A  stuffed  Trainirs  >vlce  Letter  of 
Agreement  (TDLOA}  if  the  next  phase  Is 
the  VALIDATION  PHASE  or  o  Training 
Device  Requirement/Training  Device 
Letter  Requirement  (TDR/TDLR)  if  the 
next  phase  is  the  FSED  PHASE. 

k.  A  Concept  Formulation  Packate  (CFP). 

l.  Program  Management  Plan. 

In  general  terms,  a  concept  of  a  training 
device  Is  developed  based  on  an  identified  need.  The 
concept  must  be  validated  and  undergo  Advanced 
Development  (AD)  if  there  ore  sufficient  unresolved 
technical  risks  before  progressing  to  the  undertaking  of  an 
Engineering  Development  (ED)  effort.  If  the  technical 
risks  are  not  significant  an  ED  effort  can  be  almost 
directly  undertaken  with  only  a  minimal  VALIDATION 
PHASE  in  order  to  prepare  tlie  necessary  documentation 
to  support  the  FULL-SCALE  DEVELOPMENT  (FSD) 
PHASE. 

The  second,  or  DEMONSTRATION  VALIDA¬ 
TION  PHASE  encompasses  those  activities  necessary  to 
demonstrate  preliminary  design  and  engineering,  analyze 
trade-off  proposals,  resolve  or  minimize  logistics 
problems  already  identified  and  validate  that  the 
technical  concepts  are  achievable  and  that  the  training 
effectiveness  is  attainable. 


Figure  3  -  DEMONSTRATION  AND  VALIDATION  PHASE  (PARTIAL) 
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This  phcse  of  the  training  device  acquisition 
process  Is  the  phoae  wherein  a  Project  Director  (PD) 
becomes  the  principle  manager.  He  uses  a  team  of  highly 
qualified  personnel  from  the  divisions  of  the  PM  TRADE 
and  Navy  Training  Equipment  Center  to  assist  him  in  his 
management  activities. 

The  repetitive  process  used  during  need 
identification  and  concept  formulation  is  found  in  tne 
VALIDATION  PHASE.  The  several  studies,  analyses,  and 
assessments  made  in  this  phase  are  repetitive,  mutually 
supporting,  fully  inter-related  and  are  continuously 
updated  during  this  phase  or  are  updated  versions  of  like 
efforts  from  tire  previous  phase.  These  repetitive  and 
joint  efforts  may  be  illustrated  by  the  updating  of  the 
Coordinated  Test  Program  (CTP)  for  the  Developmental 
and  Operation  Testing  (DT-I/OT-I)  series  of  tests  and  new 
or  revised  plans  for  later  series  of  tests. 

In  general  terms,  the  training  device  concepts 
of  the  CONCEPT  PHASE  are  developed  into  breadboard 
models.  They  ore  then  tested  to  demonstrate  both  tne 
validity  of  the  concepts  and  that  they  can  be  fully 
developed  in  order  to  economically  fill  the  need.  The 
design(s)  selected  by  the  Government  will  be  further 
developed  during  the  next  phase  of  the  training  device 
acquisition  cycle. 

The  complete  VALIDATION  PHASE  is  not 
found  in  many  training  device  programs.  At  the  present 


time  the  technology  used  to  develop  most  training  devices 
is  in-ivand  and  the  technical  risks,  therefore,  are  minimal. 

The  FULL-SCALE  DEVELOPMENT  (FSD)  PHASE  often 
follow  the  CONCEPT  PHASE  with  a  short  VALIDATION 
PHASE. 

Coring  the  ED  Phase  the  training  device 
system  and  all  of  the  items  necessary  for  its  support  are 
fully  developed.  The  device  is  engineered,  fabricated, 
tested,  type  classified  as  appropriate,  and  a  decision  is  - 
made  as  to  the  device's  acceptability  to  enter  the 
inventory. 

i 

The  Engineering  Development  (ED)  prototype  j 

is  developed  to  where  it  can  be  tested  and  subsequently 
produced.  The  analysis  of  the  test  results  will  provide  • 

sufficient  information  to  make  a  decision  as  to  the 
usefulness  of  the  training  device  by  tire  'JS  Army.  The  j 

necessary  supporting  items  are  also  developed.  They  are 
then  tested  to  determine  their  acceptability  to  support 
the  troining  device.  At  the  same  time,  personnel  and  ■ 

equipment  requirements,  publications,  repair  parts  and 
modifications  to  doctrine  and/or  organizations  are  i! 

finalized  so  that  an  integrated  training  device  system  is 
ready  to  be  deployed. 

Certain  training  device  programs  may 
condense  this  phase  it  immediate  acquisition  is  warranted; 
e.g.,  Military  Adaptation  of  Commercial  Items  (MACI), 
non-developmental  items  or  assemblages,  etc.  During  the 


developmental  and  operational  testing  (DT-II/OT-II) 
conducted  during  this  Phase,  engineering  development 
prototypes  are  used  to  arrive  at  a  decision  concerning  the 
technological  feasibility,  training  utility,  cost,  health 
hazards,  human  engineering,  value  engineering, 
producibility  (if  applicable),  training  effectiveness  and 
supportability  of  the  proposed  training  device.  The 
Project  Director  Is  the  primary  manager  with  the 
assistance  of  a  team  of  PM  TRADE  personnel. 

The  repetitive  process  found  in  the  two 
previous  phases  of  the  Training  Device  Acquisition 
process  is  again  present  In  this  phase.  The  significant 
difference  in  this  process  as  it  is  applied  to  the  activities 
of  the  FSD  PHASE  are  that  conclusions, 
recommendations,  procedures,  plans,  and  assessments  are 
completed  and  appropriately  implemented  particularly  if 
there  is  no  PRODUCTION  PHASE  to  follow. 

During  the  PRODUCTION  AND  DEPLOY¬ 
MENT  PHASE  operational  and  support  personnel  are 
trained,  training  devices  and  their  supporting  equipment 
are  procured  and  distributed  in  accordance  with 
requirements,  and  logistic  support  Is  provided. 
Maintainance  and  cost  data  are  collected,  evaluated  and 
used  for  systems  management  of  the  present  training 
device  and  similar  future  devices. 

The  Engineering  Development  (ED)  prototype 
is  duplicated  or  manufactured  and  delivered  to  the  Users. 


The  DEVA  IPR  may  have  provided  guidance  to  the  PM 
TRADE  and  TRADOC  that  will  be  cause  for  updating 
some  of  the  management  documentation.  It  so,  this  is 
accomplished  early  in  this  phase.  The  plons  and 
agreements  e.g.,  the  Materiel  Fielding  Plan,  Integrated 
Logistic  Support  Plan,  etc.)  are  coordinated,  completed 
and  implemented,  as  appropriate.  The  significant 
objectives  of  this  Phase  include  the  following: 

a.  Award  a  production  contract. 

b.  Accept  the  Initiai  Production  Facilities 
for  the  production  of  Items  to  the  prescribed  form,  fit, 
function  and  interchangeability  standards  that  have  been 
prescribed. 

c.  Receive  the  Long  Lead  Time  Items  that 
have  been  identified. 

d.  Review  and  update  the  logistic  support 
and  training  documentation  prepared  or  updated  during 
the  ED  PHASE  to  insure  compatibility  with  the  configura- 
*ion  of  the  production  items. 

e.  Audit  the  configuration  of  the  initial 
production  items  and  compare  it  to  the  production 
Technical  Data  Package. 

f.  Coordinate  and  finalize  the  Materiel 
Fielding  Plan  with  each  gaining  Major  Command  before 
the  Initial  Operational  Capability  date. 

g.  Finalize  and  implement  the  Integrated 
Logistic  Support  Plan.  The  implementation  of  this  plan  is 


Figure  5  -  PRODUCTION  AND  DEPLOYMENT  PHASE  (PARTIAL) 
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continuously  monitored  to  assure  the  availability  of 
trained  personnel,  supporting  hardware,  software, 
manuals,  Test  Measurement  and  Diagnostic  Equipment, 
repair  parts,  etc.}  to  file  gaining  command  before  or  with 
the  training  device  being  fielded.. 

h.  Plan  or'.d  conduct  production  testing 
and/or  Inspections. 

i.  Conduct  New  Equipment  Training  (NET). 

The  appearance  of  the  TDAMM  and  some  of 
its  features  are  demonstrated  in  the  above  portions  which 
have  been  extracted  from  the  entire  model.  The 
numerical  designations  are  to  some  extent  arbitrary  due 
to  several  events  happening  at  one  time.  Each  element  is 
given  the  title  of  the  event  or  effort  being  undertaken. 

The  alphanumeric  code  Is  a  ready  reference  to  the 
associated  test  which  briefly  describes  what  is  to  be 
accomplished  within  the  context  of  this  particular 
element,  its  inter-relationships  with  other  elements  and 
what  organization  is  most  responsible.  The  organizational 
elements  are  depicted  on  the  model  by  the  horizontal 
lines  which  are  each  given  an  PM  TRADE  organizational 
designation. 

The  Training  Device  Acquisition  process  has 
seldom  found  it  economical  to  fully  accomplish  every  task 
of  all  four  phases  nor  to  enter  full-scale  or  limited 
production.  It  is  of  note,  however,  that  many  of  the 
actions  associated  with  the  VALIDATION  PHASE  and  the 
PRODUCTION  PHASE  are  done  to  some  degree  in  the 
FULL  SCALE  DEVELOPMENT  PHASE  in  order  to  field 
low  risk  Engineering  (ED)  prototypes  which  represent  the 
full  quantity  of  training  devices  required.  Some  of  these 
actions  ares 

a.  The  Technical  Data  Package  may  be 
reduced  in  the  standards  to  which  it  must  be  prepared. 

b.  Test  and  cadre  personnel  are  limited  to  a 
very  limited  number  of  schools  and  school  operational  ond 
support  personnel. 

c.  The  Materiel  Fielding  Plan  is  completed 
and  tailored  to  a  single  User. 

d.  The  Engineering  Development  (ED) 
prototypes  may  be  the  sole  quantity  required. 

e.  The  use  of  contractor  support  personnel 
with  a  "buy-out11  of  Ihe  necessary  repair  pgrts  greatly 
reduces  the  provisioning  effort  ond  the  burden  on  the 
i.'gistic  system  to  support  very  limited  quantities  of  a 
training  device. 

f.  The  transition  from  the  PM  TRADE  to  a 
DARCOM  Major  Command  is  accom¬ 
plished. 

Each  action  shown  in  the  TDAMM  is 
examinated  by  the  Project  Director  to  decide  which 
actions  are  to  be  done  to  a  greoter-or-lesser  degree  as 
dictated  by  the  specific  training  device  to  be  produced. 
Each  action  must,  therefore,  be  considered  and  analyzed 
by  the  Project  Director  as  to  its  applicability  to  the 
Training  Device  Acquisition  process.  The  octions  are 
selected  to  be  included,  deleted,  expanded  in  scope  or 
reduced  in  scope  to  form  a  tailored  TDAMM  just  for  the 
training  device  to  be  managed.  The  TDAMM  is  a 
disciplined  guide  to  a  Project  Director  to  plan  his  project 
and  then  after  tailoring  it  provides  the  means  to  track  the 
status  of  his  efforts. 

From  an  overall  generic  viewpoint,  the  PRO¬ 
DUCTION  AND  DEPLOYMENT  PHASE  represents  the 
progression  of  the  acquisition  process  from  the 


FULL-SCALE  DEVELOPMENT  PHASE  to  on  initial 
Operational  Capability  date. 

APPLICATION  OF  TDAMM 

The  Training  Device  Application  Management 
Model  (TDAMM)  is  now  available  for  use  by  the  PM 
TRADE  to  anticipate  training  device  developments  to 
forecast,  plan,  test  and  execute  a  development  of  any 
training  device  or  significant  modification  to  a  device. 
The  model  is  also  useful  to  any  contractor  developing  a 
training  device,  it  is  applicable  to  ail  training  device 
acquisition  programs.  Moreover,  the  model  is  applied  to  a 
system-unique  as  well  as  a  non-system-unique  device.  It 
is  an  easily  recognizable  fact  that  not  every  training 
device  requires  oil  of  the  events  depicted  be  performed. 
As  the  situation  warrants,  the  acquisition  requirement  for 
a  particular  training  device  can  be  logically  selected  and 
a  tailored  version  of  the  TDAMM  prepored.  As  on 
example,  the  Conduct-of-FIre  Trainers  (COFT)  for  the 
present  Main  Battle  Tonk  and  the  future  MBT  along  with 
the  Fighting  Vehicle  series  were  severly  modified  or 
tailored.  The  TDAMM  was  initially  applied  when  the 
development  program  of  the  COFT  devices  had  nearly 
completed  one  year  of  the  three  years  Full-Scale 
Development  effort.  The  development  strategy  was  to 
use  a  "skunk  works"  method  of  development  wherein  two 
contractors  were  in  competition  to  demonstrate  their 
development  of  a  COFT  in  a  shoot-off.  This  strategy  has 
lead  to  the  avoidance  of  developing  much  of  the 
documentation  normully  required  at  this  stage  of 
development;  e.g.,  no  Technical  Data  Package  (TDP)  was 
delivered.  Under  such  circumstances,  dependent 
activities  (e.g.;  the  lack  of  a  TDP  delays  the  dependent 
Physical  Configuration  Audit)  must  be  rescheduled  within 
the  phase  or,  as  in  the  case  of  the  COFT,  delayed  until 
the  next  phase,  in  the  COFT  development  program,  the 
model  indicated  the  events  that  had  been  neglected  or 
overlooked  for  one  reason  or  another.  Many  have  had  to 
be  done  in  order  to  be  ready  for  the  Development 
Acceptance  In-Process  Review  (DEVA  IPR). 

The  application  of  the  TDAMM  to  the  troiner 
device  developments  of  PM  TRADE  illustrates  its  utility 
to  guide  management  in  its  determination  which  actions 
are  applicable  leading  to  a  system-unique  tailoring  of 
TDAMM.  Caution,  however,  must  always  be  taken  to 
ensure  that  none  of  the  202  events  is  eliminated  or 
reduced  in  scope  so  that  there  is  an  adverse  effect  upon 
the  Government  such  that  an  unsatisfactory  design  results 

Other  developmental  agencies  within  DoD  and 
foreign  countries  have  indicated  interest  in  TDAMM  to 
use  in  their  developments.  When  the  requirement  for  a 
training  device  is  received  a  small  project  team  is  formed 
in  order  to,  ornong  other  things,  prepare  a  program  plan. 
As  port  of  this  effort  the  TDAMM  is  reviewed  and 
tailored  to  manage  the  developemnt  of  the  project.  The 
dates  that  the  events  are  to  be  accomplished  are 
established  in  order  to  track  the  project.  The  program 
plan  ond  the  tailored  TDAMM  are  the  baseline 
management  tools  turned  over  to  the  Project  Director 
who,  with  a  team  of  functional  experts,  will  manage  the 
program. 

Each  training  device  project  is  managed  in  a 
like  manner.  All  of  the  training  device  develops  are 
periodically  reported  to  the  PM  TRADE  in  accordance 
with  the  individual  program  plans  and  the  tailored 
TDAMM.  In  this  way  the  PM  TRADE  is  kept  informed  of 
the  status  of  each  program. 

The  TDAMM  is  a  model  to  use  in  order  to 
forecast,  plan,  test  and  execute  the  development  of  any 
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device.  When  the  model  of  each  Individual  device  is 
Integrated  into  those  of  the  other  devices  the  total 
workload  of  PM  TRADE  Is  available.  While  this  Is  tedious 
when  done  manually,  the  automated  system  will  present 
to  the  PM  TRADE  his  total  workload  and  its  status 
virtually  Instantly  and  continuously. 


SUMMARY 

The  planning,  management  and  execution  of 
the  202  events  necessary  for  a  complete  training  device 
acquisition  program  are  complex  In  their  inter¬ 
relationships.  In  addition  to  the  complexity  of  the 
acquisition  process  Itself,  is  the  added  complexity  of  the 
organization  and  number  of  projects  within  the  purview  of 
the  PM  TRADE.  The  complete  Training  Device 

Acquisition  Management  Model,  has  been  designed  to 
logically  inter-relate  all  of  the  events  to  be  considered 
and  undertaken  by  the  PM  TRADE  as  it  develops  and 
supports  the  Army's  training  devices.  Based  on  the  best 
available  data  of  any  specific  training  device  program  the 
TDAMM  is  entered  and  a  project  management  plan  can  be 
prepared.  The  actual  pacing  factor  for  training  device 
acquisition  programs  is  the  true  progress  of  the  evolving 
device,  support,  and  associated  technical  data  package 


rather  than  any  administrative  requirements.  When  it 
appears  to  be  appropriate  to  the  PM  TRADE,  phases  may 
be  eliminated,  contracted,  or  re-sequenced  to  meet  the 
requirements  of  a  specific  device.  This  tollorlng-of-the- 
TDAMM  capability  recognizes  the  uniqueness  of  each 
training  device  development  program  while  still 
considering  all  of  the  standard  action  that  must  be  taken 
in  any  development. 


ABOUT  THE  AUTHORS 

Mr.  Eugene  J.  McGinnis  is  the  Assistant  Vice- 
President  of  the  Orlando,  Florida,  Office  of  Science 
Applications,  Incorporated.  He  manages  through  a  group 
of  Directors  the  technical  and  software  support  of  the 
U.S.  Army's  Project  Manager  for  Training  Devices  and  to 
a  lesser  degree  the  U.S.  Navy's  Naval  Training  Equipment 
Center. 

Mr.  Hoyt  (Mike)  Hammer  is  an  Integrated 
Logistics  Support  Manager  and  Systems  Analyst  who 
designed  the  TDAMM  under  the  guidance  of  Mr. 
McGinnis.  He  has  had  the  unique  experience  of  working 
for  TRADOC  (the  Army's  User  Representative)  while 
being  stationed  at  Headquarters  DARCOM  (the  Army 
Developer). 


476 


POST  DEPLOYMENT  SOFTWARE  SUPPORT 


Allen  T.  Irwin 

Technical  Director,  Science  Applications,  Inc. 
3655  Maguire  Blvd.,  Suite  1 50 
Orlando,  Florida  32803 


ABSTRACT 

As  computer  based  training  devices  proliferate,  the  need  to  maintain  the  software  associated  with  the 
devices  will  cease  to  be  an  isolated  need  and  will  become  a  general  requirement.  Based  on  experience  with 
toctical  systems,  it  can  be  estimated  that  the  maintenance  of  the  software  associated  with  a  simulator  over 
the  life  of  the  device  will  cost  at  least  as  much  as  the  original  acquisition  cost  of  the  software.  In  this  paper 
the  procedures  and  requirements  for  software  maintenance  are  analyzed.  The  many  associated  trade-offs  are 
examined.  Based  on  the  requirements  of  post  deployment  support  the  impact  on  the  acquisition  process  is 
examined.  Recommendations  are  made  as  to  how  the  acquisition  or  development  phase  can  best  support  post 
deployment  support  activities  for  software  such  that  system  utility  can  be  maximized  and  life  cycle  costs 
minimized. 


INTRODUCTION 

In  o  recent  IEEE  tutorial  on  software  cost 
estimation,  Putnam  shows  software  maintenance  costs 
ranging  from  the  same  as  initial  acquisition  costs  to 
nearly  three  times  the  cost  of  original  software 
development.  (I)  Jensen  and  Tonies  report  figures  ranging 
from  50%  to  80%  of  total  software  budgets  bein'.,  devoted 
to  software  maintenance.  (2)  Clearly,  the  Life  Cycle 
Cost  of  a  training  device  involving  software  will  include 
significant  expenditures  on  the  support  of  the  software 
after  it  has  been  deployed  for  operational  use.  Thus, 
proper  planning  for  post  deployment  software  support 
offers  the  training  device  purchaser  at  least  as  effective 
control  over  life  cycle  costs  as  does  proper  planning  for 
the  development  of  the  software.  Indeed,  as  the  expected 
useful  life  time  of  a  training  device  gets  longer,  more  and 
more  leverage  will  be  available  through  control  of  post 
deployment  costs.  This  paper  attempts  to  describe  the 
nature  of  post  deployment  software  support  activities,  the 
nature  of  the  resources  necessary  to  accomplish  the 
activities  and  to  provide  some  indication  of  the  planning 
considerations  to  be  addressed  during  system  acquisition. 

SUPPORT  ACTIVITIES 

The  need  for  post  deployment  changes  to  software 
come  from  such  sources  as: 

o  Residual  errors  not  previously  discovered, 
o  Inadequate  design  leading  to  dissatisfaction 

with  performance. 

o  Increased  user  sophistication  leading  to 

extension  of  performance  requirements, 
o  Modifications  to  the  basic  system  leading  to 
new  training  requirements, 
o  Changes  in  tactical  doctrine  also  leading  to 
new  training  requirements, 
o  Upgrading  of  trainer  hardware  leading  to  new 
interface  requirements. 

All  of  the  above  sources  generate  a  continuing  need 
to  change  or  correct  the  software  of  a  training  device 
over  its  life  time.  The  provision  for  accomplishment  and 
management  of  these  changes  in  a  cost  effective  manner 
needs  to  be  a  high  priority  gool  of  those  acquiring  the 
training  device. 

The  types  of  activity  which  characterize  post 
deployment  software  support  are  illustrated  in  Figure  I. 
The  two  are  not  totally  distinct  and  that  which  begins  as 


one  type  of  activity  generally  concludes  as  the  other.  The 
two  types  of  activity  referred  to  are  immediate  problem 
response  and  on  going  scheduled  version  production.  Most 
training  devices  are  of  sufficient  importance  today,  that 
significant  error  conditions,  once  discovered,  cannot  be 
tolerated  while  the  lengthy  process  of  formal  version 
production  is  accomplished  to  correct  the  error.  Instead, 
some  form  of  immediate  action  is  required  to  bring  the 
device  back  to  a  state  that  is  ready-for-training. 
Immediate  action  is  characterized  by  focus  on  the  single 
problem  at  hand  and  often  takes  the  form  of  a  "work 
around"  that  avoids  the  unwanted  symptoms  rather  than 
curing  the  source  problem.  Such  fixes  are  temporary  in 
nature  and  are  segregated  from  the  formal  change 
package  that  will  eventually  resolve  the  underlying 
problem. 

One  of  the  most  important  aspects  of  immediate 
action  programming  is  simplicity.  The  programmer  must 
find  the  simplest  means  possible  to  avert  the  unwanted 
performance.  He  must  seek  to  minimize  his  impact  on 
the  system.  If  he  does  not,  he  will  most  likely  introduce 
two  or  three  new  problems  for  each  one  he  resolves. 
Some  times  he  will  find  an  obvious  coding  error  and  will 
totally  clear  up  the  problem  with  a  simple  correction. 
However,  such  errors  will  generally  be  found  and 
eliminated  early  in  the  life  of  the  system.  Problems  that 
are  discovered  past  initial  deployment  are  often  caused  by 
complex  interactions  of  code  that  are  only  apparent  now 
that  the  user  has  gained  r  level  of  sophistication  in  the 
use  of  the  training  device.  For  immediate  action,  it  may 
be  best  to  suppress  an  erroneous  output  -other  than  to 
make  changes  impacting  multiple  functional  areas  of  the 
simulation.  Very  often  the  instructor  can  easily  supply 
the  missing  information  to  the  student  allowing  training 
to  continue. 

Meanwhile,  thorough  design  analysis  of  the  reported 
problem  is  conducted  and  a  complete  correction  is 
developed,  implemented  and  tested.  Once  that  process  is 
complete,  the  new  version  of  the  software,  containing 
corrections  to  multiple  problems  and  design  enhancements 
as  well,  is  released  to  the  field.  The  process  by  which  a 
problem  report  is  processed  to  final  solution  requires 
formal  control  as  described  in  the  following  section. 

Processing  of  a  Problem  Report 

A  problem  report  is  initiated  when  user  of  a  training 
device  sees  some  feature  that  does  not  appear  correct  to 
him.  The  report  is  forwarded  through  normal  main¬ 
tenance  channels  until  it  reaches  the  support  facility 
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FIGURE  1  -  POST  DEPLOYMENT  SOFTWARE  SUPPORT  ACTIVITIES 


where  on  initial  analysis  is  performed. 

The  initial  analysis  is  technical  in  nature  and  seeks 
to  answer  the  following  questions: 

1)  Can  the  situation  described  in  the  problem 
report  be  duplicated?  If  not,  the  user  reporting  the 
problem  is  queried  for  more  information  until  the  problem 
con  be  repeated  or  is  traced  to  improper  operator  action. 

2)  Is  the  system  performing  in  accordance  with 
specifications?  If  system  performance  does  not  match 
specified  requirements,  corrective  action  is  explored. 

3)  If  the  system  Is  performing  according  to 
specification,  does  the  problem  report  indicate  a  situation 
that  requires  a  design  change?  Very  often  an  apparent 


problem,  particularly  in  a  simulation  is,  in  fact,  a  trade¬ 
off  that  was  made  to  gain  some  more  important  area  of 
performance.  For  example,  a  simulation  may  have 
sacrificed  full  fidelity  of  some  feature  to  maintain  real¬ 
time  operation.  Such  past  decisions  must  be  a  matter  ot 
record  to  avoid  unnecessary  analysis  each  time  that  a  new 
user  detects  the  apparent  discrepancy.  On  the  other 
hand,  continued  reports  of  such  a  situation  may  be  the 
basis  for  a  re-evaluation  of  the  design,  indicating  that  the 
apparent  training  impact  may  have  been  more  severe  than 
originally  anticipated. 

The  initial  analysis,  therefore,  establishes  if  a 
problem  really  exists,  and  if  it  does,  whether  its 


resolution  requires  only  corrective  action  or  If  it  requires 
a  design  change. 

Once  the  initial  analysis  has  established  the  required 
resolution,  the  necessary  corrective  action  or  design 
change  Is  scheduled  for  implementation  in  a  forthcoming 
version.  Based  on  the  required  effort,  the  urgency  of  the 
change  and  the  available  resources,  one  of  the  upcoming 
version  releases  will  be  selected  for  Incorporation  of  the 
new  code.  If  a  minor  correction  of  a  few  Instructions  is 
all  that  is  needed,  it  will  most  likely  be  incorporated  Into 
the  next  release.  However,  each  version  has  a  "freeze 
date"  beyond  which,  no  further  modifications  can  be 
incorporated.  Actions  that  require  significant  design 
effort  may  be  scheduled  for  further  analysis  before 
actually  heing  scheduled  for  Implementation.  An 
important  aspect  of  the  analysis  at  this  time  is  to 
establish  "user"  concurrence  on  the  proposed  design 
change.  Only  after  the  user  has  validated  the  new 
requirement  can  it  be  scheduled  for  incorporation  in  the 
system. 

Prior  to  scheduling  for  implementation,  any  design 
change  must  be  analysed  to  develop  a  proposed  approach, 
a  probable  cost  and  an  expected  impact  on  system 
performance.  This  is,  of  course,  standard  ECP  procedure. 
The  process  leading  to  formal  scheduling  for 
implementation  is  illustrated  in  Figure  2. 

The  "Version"  Concept 

Systems  containing  relatively  little  software  require 
only  informal  means  of  control.  The  trend  todoy, 
however,  is  toward  large,  complex  software.  Such 
systems  require  formalized  baseline  control  and 
disciplined  change  procedures.  Formal  procedures 
become  necessary  whenever  the  software  is  too  large  or 
complex  to  be  mainiained  by  one  or  two  people.  Formal 
control  is  also  needed  when  the  software  is  one  system  of 
several  being  maintained  by  the  same  people  at  a  central 
facility. 

The  primary  means  of  obtaining  formal  control  over 
software  has  been  by  means  of  Version  Releases. 
Periodically,  over  the  operational  life  of  the  system,  new 
versions  of  the  software  are  released  for  operational  use. 
The  frequency  of  version  releases  varies  with  the 
criticality  of  the  system,  the  maturity  of  the  software, 
and  the  dynamics  of  the  system. 

Critical  systems,  such  as  operational  air  defense 
systems  may  generate  frequent  version  releases  to  insure 
continued  peak  performance  of  the  sys'em.  As  software 
matures,  fewer  and  fewer  latent  errors  are  present,  and 
version  releases  can  be  further  apart  without  adversely 
impacting  the  system.  Some  systems  will  always  be 
highly  dynamic.  S'  nulations  of  new  aircraft,  for  example, 
will  be  in  a  constant  state  of  change  until  the  aircraft 
they  represent  mature  and  stabilize  into  a  fixed  version. 
For  most  systems,  software  version  releases  will  be 
frequent  during  initial  operations  and  will  taper  off  to  an 
as-needed  basis  as  the  system  matures. 

Just  what  is  a  version?  Since  it  is  released  for 
operational  use,  it  obviously  must  have  undergone  all  the 
controls  and  checks  and  tests  that  any  new  item  must 
undergo  before  it  is  released  to  the  field  for  operational 
use.  A  software  version  must  be  thoroughly  validated  for 
correctness  of  design  and  implementation  as  well  as  for 
correctness  of  technical  performance  before  it  cun  be 
released.  Further,  all  ncessary  items  needed  to  support 
the  version  must  be  validated  and  ready  for  simultaneous 
distribution  to  the  field.  This  includes  all  user's  manuals, 
performance  aids,  technical  documentation,  maintenance 
procedures  and  instructions  and  any  necessary  training 
materials.  In  situations  where  a  significant  design  change 
is  being  implemented,  training  teams  must  be  prepared  to 


install  the  new  version  at  each  site  and  to  train  the 
operational  crews  on  the  use  of  the  new  system.  As  can 
be  seen,  support  of  systems  at  multiple  sites  poses  a 
logistical  problem  for  software  as  well  as  for  hardware. 

Version  Production 

Version  production  It  the  entire  process  of  analy  !s, 
planning,  design,  implementation  and  test  necessary  to 
convert  selected  modification  requirements  Into  a 
software  version  ready  for  release  to  the  using  units.  This 
process  is  Illustrated  in  Figure  3.  To  accomplish  version 
production,  all  selected  changes  must  be  carried  through  a 
design  phase  with  appropriate  reviews  and  coordination. 
The  latter  Is  particularly  important  in  software 
maintenance  efforts.  The  tendancy  to  view  a  particular 
change  01  enhancement  as  an  isolated  problem  makes  the 
possibility  of  inadvertant  adverse  effects  on  other  areas 
of  performance  an  ever  piesent  danger.  A  thorough, 
formal  and  disciplined  process  of  review  and  coordination 
must  be  established  and  enforced  throughout  the  version 
production  process.  If  not,  the  version  will  never  survive 
the  prerelease  testing  to  qualify  it  for  operational  use. 

Design  Teams 

6ne  technique  for  accomplishing  such  coordination 
and  review  is  the  establishment  of  design  teams.  Each 
change  or  design  requirement  is  assigned  to  a  lead 
analyst.  Then  a  representative  of  each  functional  area  is 
appointed  to  form  a  design  team.  Each  member  may  well 
serve  on  several  design  teams,  because  their  primary 
function  is  to  review  the  work  of  the  lead  analyst  to 
ensure  that  Impact  on  their  functional  area  of 
responsibility  is  correctly  understood.  The  lead  analyst  is, 
of  course,  selected  from  the  functional  area  most 
severely  effected  by  the  change.  It  is  his  responsibility  to 
coordinate  his  design  with  all  members  of  his  team  and  to 
obtain  formal  concurrence  from  the  team  that  this  design 
is  an  acceptable  implementation  for  all  functional  areas. 

The  design  team  also  serves  to  coordinate  the 
documentation  and  testing  associated  with  the  particular 
change  package.  Changes  to  user  manuals,  technical 
documentation,  maintenance  manuals,  and  training 
materials  must  all  be  prepared  and  coordinated.  Tests 
must  be  prepared  that  will  test  out  the  chcnge  itself  and 
also  to  validate  that  nil  other  areas  are  not  impaired. 

Version  Testing 

Version  testing  mirrors  normal  software  develop¬ 
ment  testing,  but  with  some  significant  differences.  For 
modifications  representing  new  design,  a  bottom  up 
approach  is  generally  followed.  The  modified  modules  are 
tested  and  debugged  in  isolation  then  integrated  into  the 
system  and  tested  for  correct  operation  at  the  system 
level.  Smaller  corrections  may  only  require  verification 
at  the  system  level  to  show  that  they  have,  in  fact,  been 
correctly  implemented.  The  most  significant  fact  about 
version  testing  is  that  any  one  version  changes  only  a  very 
small  percentage  of  the  system  software  perfoimance 
requirement.  Thus,  system  level  version  testing  does  not 
require  a  new  system  test  fer  each  version,  but  rather  a 
new  version  of  the  already  validated  system  test.  This 
approach  to  testing  can  allow  significant  savings  in  the 
testing  effort  required  1o  validate  a  new  version  for 
release.  A  very  desirable  approach  is  to  develop  an 
automated  system  test  that  provides  stimulating  inputs 
and  automatic  analysis  of  results  (See  Figure  4).  Such  a 
test  tool,  once  accepted  by  the  user  can  greatly  reduce 
the  effort  of  version  qualification.  The  automated  test 
program  can  short  circuit  normal  operator  input  and 
output  since  it  is  required  to  show  only  that  unmodified 
software  still  performs  correctly.  Thus,  by  running  pre¬ 
recorded  inputs  through  the  system  and  automatically 
comparing  the  outputs  with  expected  results,  it  validates 
the  operation  of  the  software  not  affected  by  the  changes 
incorporated  by  the  new  version.  (The  system  test  itself, 
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FIGURE  2  -  PROBLEM  REPORT  PROCESSING 


must  be  updated  for  each  version,  however,  this  is  much 
more  manageable  than  developing  a  new  test  each  time.) 

The  test  cycle  for  the  version  is  as  follows: 
o  Debugging  at  the  module  level 
o  Testing  of  new  features  for  correct 
implementation  at  the  system  level 
o  Validation  of  unmodified  software  using 
the  automated  system  test 
o  User  (Operational)  Testing  of  new 
features 

An  important  consideration  of  the  last  step  in  the 
above  sequence  is  how  much  user  testing  is  required? 
Clearly,  the  user  of  the  training  device  must  be  satisfied 
that  it  is  performing  correctly.  However,  it  should  not  be 
necessary  to  revalidate  by  operational  testing  all 


performance  parameters  of  the  system  at  every  version 
release.  The  ability  to  satisfy  the  user  (he  is,  after  all, 
the  customer)  that  he  needs  only  test  the  new  features  of 
the  version  will  depend  heavily  on  the  degree  to  which  the 
user  can  be  convinced  that  the  automated  version  test 
does  successfully  validate  the  unchanged  software.  This 
is  further  support  for  a  weil  planned  and  well  documented 
system  tost. 

Formal,  centralized  control  is  a  necessity  in  the  test 
program.  If  central  control  is  not  maintained,  testing  can 
degenerate  into  an  unending  set  of  failures  as  test  after 
test  encounters  unexpected  side  effects.  This  can  only  be 
avoided  if  proper  resources  are  committed  to  the  te$>ing 
effort  throughout  version  production.  Testing  is  also  very 
sensitive  to  "Version  Freeze  Date".  If  the  input  of  change 


480 


INPUTS 


ACTIONS 


PRODUCTS 


•  PERFORMANCE  REQUIREMENTS 

•  TESTING  REQUIREMENTS 

•  RESOURCE  REQUIREMENTS 


•  SCHEDULES 

•  PRIORITIES 

C  RESOURCE  ALLOCATION 


•  DETAILED  SPECIFICATIONS 

•  TEST  PROCEDURES 

•  OPERATIONAL  PROCEDURES 


•  REVISED  CODE 

•  REVISED  TEST  SOFTWARE 

•  REVISED  DOCUMENTATION 


•  TEST  RESULTS  , 

•  MODIFICATIONS 

•  VALIDATED  PERFORMANCE 
AND  DOCUMENTATION 


•  CERTIFICATION  FOR 
OPERATIONAL  USE 


•  SYSTEM  OPERATING 
UNDER  NEW  VERSION 


FIGURE  3  -  THE  VERSION  PRODUCTION  PROCESS 


requirement  for  a  particular  version  are  not  cutoff  early 
enough,  testing  will  suffer  along  with  design.  This  can  be 
offset  by  scheduling  version  release  dates  with  sufficient 
frequency  to  reduce  the  pressure  to  keep  adding  "just  one 
more  change". 

Scheduling  of  Version  Releases 

The  time  required  to  design,  develop,  implement, 
test  and  release  a  version  will  necessarily  vary  with  its 
contents.  If  the  version  is  primarily  to  correct  coding 
errors  then  it  will  require  o  short  period  to  prepare  it  for 
release.  As  new  design  effort  is  added  to  the  version's 
contents,  the  required  time  length  will  go  up.  Thus, 
version  release  may  take  from  6  to  18  months.  When 
significant  hardware  changes  are  also  involved,  the 
release  time  could  easily  approach  the  initial  development 


schedule  and  run  as  much  os  3  years. 

For  administrative  reasons,  it  may  prove  desirable 
to  schedule  versions  for  release  on  a  fixed  timetable.  In 
this  case,  version  content  can  be  adjusted  to  meet  the 
schedule.  In  addition  the  start  date  for  versions  selected 
for  significant  change  efforts  can  be  set  ahead  of  the 
normal  version  start  time.  The  phased  nature  of  version 
production  lends  Itself  to  overlapped  production  of 
versions.  That  is,  while  one  version  is  undergoing  testing 
by  the  testers,  a  second  one  can  be  in  the  hands  of  the 
programmers,  while  system  analysts  have  begun  the 
design  of  a  third  version.  In  this  manner  three  versions 
are  In  production  at  any  one  time.  Version  release  occurs 
more  frequently  and  the  technical  staff  Is  allowed  to 
specialize  in  the  three  specific  areas  of  design, 
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AUTOMATED  TESTING  PROCESS 


implementation  and  testing.  Such  specialization  leads  to 
increased  efficiency  since  each  area  requires  particular 
skills  and  not  many  individuals  will  be  equally  capable  at 
all  three  areas. 

PLANNING  FOR  POST  DEPLOYMENT 
SOFTWARE  SUPPORT 

In  establishing  the  Post  Deployment  Support 


Concept  for  a  training  device,  questions  such  as  where 
software  support  will  be  performed,  who  will  accomplish 
the  support,  and  what  equipment  will  be  required  must  be 
answered.  The  answers  to  such  questions  are  not 
arbitrary,  but  are  determined  by  several  contributing 
factors. 
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Locating  the  Software  Support  Capability 

Assuming  that  wt  art  talking  about  a  training 
device  with  u  significant  software  support  requirement, 
then  the  Issue  of  where  to  perform  the  support  task  Is 
largely  a  question  of  resources.  Perhaps  one  of  the 
primary  considerations  is  the  availability  of  equipment. 

It  does  no  good  to  locate  the  maintenance  facility  at 
the  training  site  If  the  training  device  Is  so  heavily 
committed  to  training  use,  that  no  time  Is  available  for 
software  maintenance  usage.  The  temptation  to  save 
money  by  planning  to  do  all  software  maintenance  on 
operational  equipment  must  be  balanced  against  the  very 
real  cost  penalties  of  having  programmers  being  paid  to 
sit  around  waiting  for  computer  time.  The  early  planning 
for  training  devices  generally  Includes  an  assessment  of 
device  utilization.  Using  this  assessment  plus  an  estimate 
of  how  much  device  time  will  be  required  for  hardware 
maintenance,  provides  an  indication  of  what  time  will  be 
left  over  for  software  maintenance  utilization.  Analysis 
of  the  software  maintenance  task,  based  on  the  dynamics 
of  the  system  that  will  lead  to  changes  to  the  training 
device  can  provide  an  estimate  of  the  size  of  the  software 
workload  and  hence  the  required  device  access.  If  a 
conflict  exists  as  to  device  availability,  then  the  thought 
of  using  operational  equipment  should  not  be  carried 
further  since  the  Initial  estimates  of  software  change 
activity  usually  tend  to  be  low! 

Once  the  decision  is  made  to  acquire  a  software 
support  facility  (l.e.,  computer  equipment  dedicated  to 
software  maintenance)  the  question  of  where  to  locate  It 
still  must  be  resolved.  At  this  point  the  other  essential 
resource  must  be  considered— personnel.  If  a  centralized 
support  facility  already  supporting  similar  or  related 
training  devices  exists,  then  it  provides  an  attractive  base 
for  economically  expanding  to  provide  support  for  the  new 
system.  If  a  stand-alone  maintenance  facility  U  to  be 
established,  it  will  generally  require  more  personnel  than 
would  be  required  as  add-ons  to  an  existing  facility.  An 
existing  facility  also  provides  a  source  of  personnel 
experienced  in  the  technical  and  managerial  skills 
associated  with  software  maintenance.  (It  should  be 
recognized  here  and  now  that  the  developing  contractor  Is 
not  going  to  turn  over  his  design  and  programming  staff  to 
maintain  the  delivered  equipment.  They  will  be  off 
designing  and  programming  new  products.)  A  brand  new 
software  support  facility  will  always  undergo  an  expensive 
learning  curve  as  the  new  personnel  come  up  to  speed 
with  the  system  and  with  the  maintenance  process. 

The  equipment  placed  in  the  support  facility  need 
not  necessarily  be  as  elaborate  as  the  full  up  training 
device.  The  majority  of  software  development  and 
testing  can  be  accomplished  without  the  full  set  of 
hardware  required  for  the  more  elaborate  training 
devices.  While  training  time  will  not  usually  permit 
software  maintenance  to  be  fully  supported  on  operational 
equirnent,  it  usually  can  support  some  of  the  final  system 
testing  necessary  for  version  release.  Thus,  a  concept  of 
shared  facilities  may  prove  a  useful  compromise  with  the 
majority  of  software  maintenance  being  accomplished  at 
a  less  elaborately  equipped  support  facility  and  the  formal 
system  level  version  acceptance  testing  being 
accomplished  at  a  designated  operational  "test"  site. 

When  choosing  the  operational  site  or  an  offsite 
facility  location,  consideration  should  be  given  to  the  ease 
with  which  qualified  personnel  can  be  recrui+ed 
andretalned  at  the  locations  under  consideration. 
Questions  such  as  the  availability  of  trained  personnel, 
and  the  desirability  of  the  geographical  area  will  have 
long  term  significant  economic  Impact  on  the  operation  of 
the  facility. 


The  "WHO"  of  Software  Support 

Daley,  speaking  of  commercial  software,  estimates 
that  one  full  tl  ,ie  programmer  can  maintain  10,000  lines 
of  realtime  so  (ware  or  30,000  lines  of  support  software. 
(3)  This  Igure  does  not  Include  configuration 
management,  lupervlslon,  field  support  or  other  overhead. 
While  his  estln  aie  Is  not  necessarily  directly  applicable  to 
training  devlct  software,  it  does  serve  to  illustrate  that 
the  manpower  requirements  to  maintain  any  significant 
software  package  are  not  small. 

As  Daley  also  points  out,  a  programmer  will  be  more 
effective  In  a  maintenance  roll  If  half  of  his  time  is 
devoted  to  maintenance  and  half  to  development  of  new 
software.  (3)  This  effect  derives  partly  from  motivation 
and  partly  from  proficiency  maintenance  gained  by 
experience  with  new  systems.  This  factor  seems  to  argue 
against  dedicated  software  support  facilities.  However, 
the  experience  of  developing  new  software  can  be 
provided  at  a  support  facility  If  system  enhancements  are 
also  assigned  to  the  facility  for  development.  Otherwise, 
It  can  be  expected  that  a  facility  devoted  solely  to  error 
correction  and  minor  enhancements  will  suffer  as  good 
people  move  on  to  more  Interesting  (and  better  paying) 
work  and  those  that  remain  lose  touch  with  the 
developmental  side  of'  software  programming. 

The  question  of  contractor  support  versus 
Government  in-house  support  of  software  is  frequently 
raised.  There  Is,  however,  no  simple  one  time  answer  to 
the  Issue.  The  choice  must  be  made  and  justified  on  a 
system  by  system  basis. 

The  key  Issue,  If  Government  in-house  support  is 
being  considered  very  often  comes  down  to  whether  or  not 
the  Government  can  make  available  the  personnel 
resources  needed  to  maintain  the  software.  Right  now 
the  Government  has  limited  capabilities  In  this  area, 
however,  as  tactical  computer  systems  proliferate,  the 
software  support  capabilities  within  DoD  will  expand 
dramatically.  Thus,  It  may  be  that  In  a  few  years, 
assignment  of  training  device  software  to  a  Government 
software  support  facility  will  be  routine. 

The  possibility  of  Government  support  of  the 
software  at  some  future  date  is  another  argument  for  the 
acquisition  and  maintenance  of  thorough  and  complete 
documentation  of  aM  software,  even  if  it  is  to  be  initially 
supported  by  the  developing  contractor. 

Post  Deployment  Software  Support  Equipment 

Wien  acquiring  a  training  device,  considerable 
thought  should  be  given  to  the  acquisition  of  software 
support  equipment.  As  mentioned  above,  operational 
equipment  is  often  not  available  for  software  support. 
This  makes  it  necessary  to  have  computer  equipment 
dedicated  to  the  support  role.  One  means  of 
accomplishing  this  is  to  assign  prototype  equipment  to  the 
support  role  upon  completion  of  prototype  testing. 
Another  means  is  to  buy  excess  computer  power  so  that 
both  operational  and  support  functions  can  be 
accommodated  in  a  timesharing  mode.  (This  Is  only 
possible  where  the  operational  requirements  can  tolerate 
the  limitations  of  time  sharing  operation.)  In  many  cases 
It  will  be  necessary  to  buy  (or  lease)  a  totally  separate  set 
of  equipment  for  the  support  role. 

Whichever  method  is  selected,  It  is  important  that 
requirements  for  the  support  function  aro  specified  and 
designed  in  from  the  beginning.  Attempts  to  go  back  and 
force  such  functions  In  later  on  always  meet  with  limited 
success  and  are  generally  much  more  costly  than  when 
they  are  recognized  and  designed  in  from  the  beginning. 
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When  the  prototype  of  a  training  device  Is  to  end  up 
as  the  software  support  facility,  thought  should  be  given 
to  procuring  excess  computer  power  In  the  prototype 
configuration.  Not  only  will  this  make  the  development 
of  the  operational  software  easier,  it  can  also  allow  the 
support  facility  to  support  multiple  programmers  In  a 
time-sharing  mode.  It  will  also  make  It  possible  to 
establish  a  test  environment  for  the  operational  software 
by  the  addition  of  other  software  designed  to  provide  on¬ 
line  analysis  of  operational  software  performance. 

SUMMARY 

In  conclusion,  the  planners  of  a  training  device 
development  can,  through  proper  provisions  for  post 
deployment  software  support,  do  as  much  or  more  to 
control  the  overall  cost  of  the  training  device  as  can  be 
accomplished  by  proper  provision  for  the  Initial  software 
development.  Coversely,  failure  to  consider  support 
requirements  for  software  can  be  just  as  dlsasterous  as 
failure  to  consider  hardware  support  requirements.  Lack 
of  adequate  provision  for  software  support  can  prevent 
effective  use  of  the  training  device  and  can  cause 
excessive  growth  of  life  cycle  costs. 

Software  support,  once  the  training  device  Is 
deployed,  will  consist  of  software  modifications  to 
enhance  device  performance,  to  eliminate  latent  errors, 
or  to  reflect  changes  in  the  operational  equipment. 

These  modifications  must  be  managed  to  ensure  that 
the  operational  validity  of  the  training  device  is  not  lost. 
Thus,  formal  procedures  must  be  established  for  the  post 
deployment  support  operations. 

Two  primary  activities  characterize  post 
deployment  support  operations!  Immediate  action  and 
version  production.  Immediate  action  is  Intended  to 
provide  quick  fixes  or  work-arounds  for  problems  detected 
during  system  operation.  Version  production  is  the  formal 
process  of  analysis,  planning,  design,  Implementation, 
testing,  release  and  installation  that  provides  integrated 
and  validated  software  enhancements  and  fault 
corrections  that  ensure  effective  and  efficient  use  of 
system  resources  throughout  its  life  cycle. 

To  make  certain  that  the  support  facilities  and 
resources  required  for  post  deployment  software  support 
are  ready  and  in  place  when  needed,  planning  must  begin 
with  the  initial  training  device  concept  formulation.  To 
accomplish  the  necessary  planning,  consideration  must  be 
given  to  where  the  software  support  activity  will  be 
accomplished;  what  personnel  resources  will  be,  a) 
required,  and  b)  available  to  provide  the  support;  What 
equipment  is  to  be  used  for  post  deployment  support  and 
how  it  is  to  be  obtained,  and  how  the  overall  effort  is  to 
be  managed. 

It  Is  strongly  recommended  that  a  Post  Deployment 
Software  Support  Plan  be  developed  early  in  the 
acquisition  process  and  that  it  be  maintained  up-to-date 
throughout  the  device  life  cycle. 
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ABSTRACT 

(  The  paper  sunmarlzes  the  implications  of  the  shrinking  U.S.  person-power  base  In  the 
8C's  and  90's,  Including  the  competition  for  these  resources  by  the  government,  military, 
and  Industry  sectors.  Probably  the  most  practical  solutions  lie  in  designing  as  many  O&M 
personnel  out  of  the  weapon  systems  as  practical;  and  Increasing  the  productivity  of  those 
remaining  personnel  via  Improved  training-programs.  Four  key  joint  DoD  and  Industry  actions 
can  facilitate  these  solutions;  a)  Increase  first  term  productivity;  b)  improve  career 
selection;  c)  emphasize  transfer  of  training  studies;  and  d)  reduce  personnel  requirements 
based  on  credible  personnel  subsystem  life  cycle  cost  modeling.  The  technology  exists. 


THE  PROBLEM 

Personnel  shortages  will  exist.  Figure  1 
portrays  the  now-familiar  demographic  fact  that 
the  number  of  18  to  24  year  old  males  entering 
the  workforce  will  decrease  by  a  significant 
amount  over  the.  next  16  years.  This  represents 
a  cross-section  of  personnel,  from  unskilled 
laborers  through  professionals. 


WORKFORCE  GROWTH 

"THE  WARNING  SIGNS 
OF  A  LABOR  SHORTAGE" 


(MILLIONS! 


Figure  1:  The  Traditional  Source  of  Manpower 
Base  Will  Not  Keep  Up  with  Demands 

This  shortage  of  available  manpower  is 
beginning  to  be  felt  right  now.  It  is  a  problem 
that  requires  Immediate  solutions,  because  people 
are  long  lead  time  Items:  you  can't  grow  an 
18  -  24  year-old  overnight. 

The  personnel  shortage  affects  Industry 
(and  the  generation  of  the  GNP  which  pays  for 
government),  as  well  as  government  and  the  mili¬ 
tary.  Industry  demands  are  Increasing,  especially 
for  the  higher  skill  categories.  For  example, 
the  electronics  Industry  Is  growing  at  about  17% 
per  year  according  to  some  recent  statistics. 


The  second  major  user  of  manpower,  the  govern¬ 
ment  sector,  has  vowed  that  it  is  cutting  back 
Its  requirements.  However,  Increased  Industry 
requirements  will  probably  counterbalance  govern¬ 
ment  cutbacks.  The  projected  military  manpower 
demands  over  the  1980  -  2000  time  period  are 
relatively  constant  at  around  2  million  personnel. 

Since  total  manpower  requirements  of  the 
three  sectors  will  remain  constant  or  increase, 
and  the  available  workforce  will  shrink  drasti¬ 
cally,  the  military  cannot  realistically  expect  to 
satisfy  its  future  manpower  requirements  by  draw¬ 
ing  from  industry.  The  military  will  have  to  seek 
elsewhere  to  solve  Its  personnel  problems. 

Various  internal  solutions  have  been  pro¬ 
posed  to  solve  the  military  manpower  shortage 
problem;  these  Include: 

a)  later  retirement 

b)  more  use  of  women 

c)  employing  the  "unemployable" 

d)  the  draft 

Upon  examination,  later  retirement  does  not 
seem  to  provide  a  practical  solution  to  the  man¬ 
power  requirements.  As  the  military  acquires 
increasing  numbers  of  weapons  systems.  Its  greatest 
manpower  requirements  are  for  operators  and  main¬ 
tenance  personnel  to  man  the  fielded  systems.  Yet 
job  advancement  moves  more-experienced  personnel 
out  of  technical  billets  Into  supervisory  and 
adninistrative  positions.  Attempts  to  fill  many 
of  the  highly  technical  or  lower  skill-level  mili¬ 
tary  O&M  billets  with  retirement- age  personnel 
might  create  technical  manning  gaps,  create  severe 
morale  problems,  and  be  far  too  expensive  in  terms 
of  pay  and  benefits. 

Women  are  being  employed  In  the  military 
for  both  administrative  and  non-tradltlonal  tech¬ 
nical  roles.  At  one  time  this  approach  seemed  to 
offer  considerable  promise.  However,  recent 
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statistics  published  by  the  military  Indicate 
that  this  approach  will  not  fill  the  gap. 

The  percentage  Increase  of  personnel  In 
lower  mental  categories  joining  the  military 
these  days  has  been  well-publicized  In  recent 
years.  Placing  Increasing  reliance  upon  such 
personnel  to  operate  and  maintain  the  Increas¬ 
ingly-sophisticated  weapon  systems  coming  on¬ 
stream  does  not  seem  to  provide  a  sound  long¬ 
term  solution  to  the  manpower  shortage. 

Thus,  It  would  appear  that  a  more  compre¬ 
hensive  solution  to  the  Increasing  manpower- 
shortage  Is  badly  needed.  The  most-promising 
solution  Is  to  draw  markedly  Increased  produc¬ 
tivity  from  the  dwindling  military-manpower 
resources  available.  Part  of  this  Increase  can 
be  achieved  through  Improved  motivation  and 
commitment  of  the  military  workforce.  Most  of 
the  productivity  Increase  must  be  developed  by: 

(a)  reducing  the  numbers  of  people  required  to 
counter  all  anticipated  military  threats  to  the 
country;  and  (b)  Improving  the  training  of  those 
people. 

Prerequisite  to  Increasing  the  productivity 
of  military  manpower,  we  must  define  clearly  every 
single  job  or  billet  In  terms  of  Its  ‘‘productive" 
contribution  to  meeting  the  mission;  and  we  must 
develop  a  valid  means  to  evaluate  how  effectively 
the  Individuals  occupying  the  billets  meet  those 
productivity  goals.  The  measure  of  billet  pro¬ 
ductivity  may  range  from  effectiveness  In  manning 
a  system  In  combat,  to  maintaining  the  system  to 
Its  availability  and  readiness  specs,  to  restor¬ 
ing  a  high  level  of  motivation  and  enthuslam 
through  a  well-planned  and  wel 1 -executed  R  and  R 
program. 

Increasing  the  productivity  of  military 
personnel  will  have  a  drastic  beneficial  effect 
upon  the  DoD  budget.  The  cost  of  manpower  (the 
personnel  subsystem)  Is  the  most  significant  Item 
in  the  budget,  over  30%  of  the  total  budget,  and 
over  50%  of  the  0AM  budget.  These  percentages  do 
not  even  include  the  large  annual  retirement 
costs,  housing  construction,  training,  and  other 
indirect  personnel -related  costs.  When  we  devise 
ways  to  calculate  such  Indirect  costs  accurately, 
and  add  them  to  the  direct  personnel  costs,  the 
budgetary  importance  of  Improving  the  productivity 
will  be  just  as  apparent  as  that  of  reducing  the 
manpower  shortage. 

We're  at  the  critical  point.  What  can  we 
do? 


JOINT  DoD  AND  INDUSTRY  SOLUTION 

There  are  four  key  actions  that  can  be 
taken  jointly  by  DoD  and  Industry  to  solve  the 
manpower  shortage.  These  are: 

a)  Develop  methods  for  increasing  first 
term  enlistment  productivity. 

b)  Improve  seicCtlon/moli vatimi  systems 
for  potential  career  NCOs. 

c)  Fund  more  research  Into  the  cost  effec¬ 
tiveness  of  Improved  training  strate¬ 
gies  and  devices  on  transfer  of  train¬ 
ing. 

d)  Develop  simple,  fast,  credible  person¬ 
nel  subsystem  costing  techniques  keyed 
to  weapon  system  design  parameters,  to 
substantiate  designing  people  out  of 
the  system. 

Each  key  action  is  designed  to  reduce  the 
military’s  manpower  requirements.  The  first  three 
actions  will  enable  fewer  personnel  to  do  more 
work  by  means  of  increased  productivity.  The 
fourth  action  will  reduce  manpower  requirements  by 
means  of  weapons  system  design  tradeoffs. 

First  Term  Productivity 

Over  20  percent  of  the  enlisted  force  con¬ 
sists  of  first  year  enlistees.  The  total  percen¬ 
tage  of  "first  term"  enlisted  and  officer 
personnel  is  much  greater.  Therefore,  It  would 
seem  that  Investment  in  technology  to  increase 
first  term  productivity  would  reap  the  highest 
near-term  benefit.  This  might  be  envisaged  as 
"return  on  investment"  in  terms  of  additional 
productive  manhours  per-dollar- invested  In  the 
personnel  management  and  training  system.  Such 
investment  In  training  not  only  reduces  the  drain 
on  the  scarce  manpower  pool  (since  more  produc¬ 
tive-manhours  per  person  generally  means  less 
people  required,  even  allowing  for  minimum 
manning  requirements),  but  also  reduces  the 
attendant  salary  and  support  costs. 

Figure  2  shows  a  typical  first  term  person¬ 
nel  Investment  cost/productivity  return  cycle  for 
an  enlistee.  The  Investment  cost  consists  of 
recruiting,  training,  and  support.  The  productiv¬ 
ity  return  Is  the  performance  of  mission-related 
job  tasks. 


Figure  2:  Low  Productivity  Results  In  Military  Manpower  Problems 
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The  three  most  significant  considerations 
controlling  the  productivity  return  are: 

1)  hew  soon  after  service  entry  the 
enlistee’s  formal  or  Institutional 
training  can  be  completed  (graduation); 

2)  the  enlistee's  average  level  of  produc¬ 
tivity  at  graduation  (e.g.,  number  of 
productive  manhours  he  can  effectively 
provide  per  week); 

3)  how  soon  the  personnel  can  become  com¬ 
pletely  qualified  in  their  assigned 
specialties  (l.e. ,  pass  a  realistic, 
criterion-referenced  skill  qualifica¬ 
tion  test  or  SQT). 

There  has  been  considerable  comment— via 
symposia,  Senate  Hearings,  and  speeches— to  the 
effect  that  few  f 1 rst-tarmers  qualify  In  their 
specialty  Jurlng  their  first  enlistment.  This 
problem  adds  a  further  supervisory  load  to  the 
already  over  loaded  NCOs  in  the  field.  It  also 
gives  rise  to  the  constant,  though  contradictory, 
complaint  by  NCOs  In  all  services  that  the  schools 
make  work  for  them  by  not  providing  personnel  who 
are  qualified.  (The  contradiction  Is  that  these 
NCOs  also  harp  on  the  theme  that  "experience  Is 
the  best  teacher.") 

At  issue  Is  the  need  to  establish  a  close- 
knit,  honest,  unselfish  link  between  the  school 
and  the  field  by  means  of  comprehensive  "career" 
(first  term)  training  programs  for  each  specialty. 
The  Army  Is  making  some  headway  in  this,  possibly 
as  a  result  of  the  high  (Four- Star)  organizational 
level  of  the  policy-making  function  for  personnel, 
doctrine  and  training.  The  Navy  Air  conmunlty  is 
also  attempting  to  achieve  this  with  their  newer 
(F/A-18)  Fleet  Introduction  Team/FRAMP  program; 
coupled  with  Personnel  Qualification  Standards 
and  Fleet  Evaluation  Teams. 

The  key  to  success  of  this  program. is  a 
systems  approach  to  developing  each  individual 
specialty  training  program  covering  every  step 
from  enlistment  to  achieving  "qual"  standards. 
School  training  must  be  task-oriented,  and  care¬ 
fully  designed  for  transition  to  field  training. 
The  latter  phrase  implies  that  the  training 
material  is  designed  by  teams  of  experienced 
trainers  and  field  experts  who  can  identify  those 
tasks  best-trained  In  school  and  those  tasks  best- 
trained  In  the  field.  In  response  to  the  tradi¬ 
tional  objection  to  shifting  tasks  to  the  field. 
General  Hilsman,  Director  of  DCA,  made  two 
significant  observations  In  his  speech  presented 
to  the  NSIA-sponsored  tri-service  conference  on 
Personnel  and  Training  Factors  last  May: 

1)  we  must  train  professional  leaders,  on 
how  to  train  people  In  the  field  (en¬ 
suring  that  the  training  material  Is 
well  designed  for  use  in  the  field); 
and 

2)  we  must  teach  these  professional  lea¬ 
ders  time  management  so  that  they  can 
train  people  In  the  field. 


The  design  and  development  of  each  compre¬ 
hensive  first-term  training  program  should  be 
based  on  such  factors  as  task  criticality,  hazard, 
tlme-to-learn,  use  of  fielded  capital  assets,  war¬ 
time  versus  peacetime  scenarios,  etc. 

One  very  significant  contribution  to  task 
and  skill-oriented  training  programs  can  be  made 
by  identifying  design  and  development  parameters 
for  families  rf  low-cost  part-task  and  p^rt- 
mission  trainers  for  both  school  and  field  appli¬ 
cation.  Making  more  use  of  cost-effective  Compu¬ 
ter  Based  Training  System  (CBTS)  technology  would 
enable  both  school  and  field  personnel  to 
Individualize  the  training.  Thus  the  Instructors 
could  concentrate  their  skills  on  helping  Indiv¬ 
idual  learners  solve  Individual  problems,  rather 
than  resorting  to  time-consuming  traditional  stand- 
up  lectures.  These  factors  should  enable  the 
enlistee  to  graduate  earlier  from  the  schoolhouse, 
at  a  higher  level  of  productivity,  with  confidence 
of  achieving  full  qualification  early  In  the 
enlistment— as  depicted  in  Figure  3. 


Figure  3:  Career  Training  Approach  Means  More 
Productive  Manhours 

Significant  technical  training  system 
design  Improvements  can  be  made  In  the  schools  and 
In  the  field  with  existing  technology.  Having  the 
technical  knowledge  baseline,  or  access  to  It,  and 
the  required  skills  will  not  solve  the  productiv¬ 
ity  problem  if  the  trainers,  operators  and  maln- 
talners  do  not  have  the  "attitudinal"  skills  to 
make  the  system  work.  When  the  standard  set  of 
procedural Ized  cures  documented  In  the  TM  do  not 
solve  an  equipment  problem,  what  creative  value- 
judgement-  based  steps  can  (and  will)  the  person 
take  to  solve  the  problem?  What  attitudinal 
training  techniques  can  be  used  to  convince  him 
that  all  problems  are  solvable  and  that  the 
equipment  is  designed  to  work?  This  has  been  sub¬ 
ject  to  considerable  academic  research.  The 
technology  exists  to  support  practical,  experi¬ 
mental  research  which  Is  end-result-oriented  such 
that  it  can  be  immediately  implemented  in  service 
school  and  field  training  programs. 

Potential  Career  NCO  Selection  and  Motivation 
Systems 


The  increased  productivity  described  ear¬ 
lier  can  only  be  achieved  if  the  comprehensive 
first  term  "career"  training  programs  are  Imple¬ 
mented  by  experienced,  competent,  motivated 
career  NCOs.  The  problem  is  how  can  prospective 
career  NCOs  be  detected,  selected,  and  motivated 
during  their  first- term  enlistments,  and  convinced 
to  stay  in  the  service?  The  selection  system  must 
be  criterion- referenced  to  identify  as  potential 
NCOs  those  personnel  with  the  best  career  poten¬ 
tial,  The  selection  system  must  be  designed  to 
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resist  challenge  by  other  personnel. 

The  services  might  consider  borrowing  from 
industry  some  of  the  techniques  used  to  spot  and 
develop  project  leaders  and  more  junior  super¬ 
visory  personnel,  particularly  In  the  more 
technical  fields.  This  Involves  tracking  and 
evaluating  promising  personnel  early  In  their 
careers  In  such  areas  as  technical  ability,  de¬ 
sire  to  lead  or  manage,  and  willingness  to  learn 
supervisory  and  managerial  techniques.  This 
should  be  accomplished  by  experienced  NCOs  and 
officers,  using  semi-standardized  (but  not 
procedural  1 zed)  matrices  of  leadership  character¬ 
istics  for  guidance.  Assignment  of  progressively 
more  responsible  tasks  on  the  job,  Including 
tenporary  lead  functions,  permits  observation  of 
individuals'  performance. 

This  is  the  critical  factor.  Because  of 
short  enlistment  times  and  perhaps  several 
assignments  (recruit,  depots,  schools,  etc.) 
early  In  the  first  enlistment.  It  Is  otherwise 
nearly  Impossible  to  adequately  observe  a  first- 
term  enlistee's  performance,  and  evaluate  his 
potential . 

If  the  grooming  and  motivation  is  not 
started  early  In  the  first  term  enlistment—  if  It 
is  left  until  the  pre-discharge  career  counseling 
sessions- -then  promises  of  bonusss,  schools, 
challenging  assignments,  etc.,  will  likely  fall 
on  deaf  ears,  for  the  short  timer  will  have 
already  made  his  decision  to  leave  the  service. 

Also,  from  an  economics  viewpoint,  the 
services  must  be  selective  In  Identifying  and 
wooing  their  future  NCOs.  For  example,  the  FY81 
retired  pay  for  the  U.S.  Army  alone  was  over  10% 
of  the  total  Army  budget. 


Increased  Government  Sponsored  Transfer  of 
Training  Studies 


Although  much  research  has  been  done  into 
media  selection  models  and  transfer  of  training 
effectiveness,  particularly  In  an  academic  light, 
conditions  are  now  ripe  for  some  concentrated 
practical  research  and  experimentation  on  a 
broader,  DoD-sponsored  scale.  Instructional  sys¬ 
tem  design  and  training  methodologies  have  been 
advanced  considerably  In  recent  years.  Some  of 
the  lamer  aerospace  companies,  involved  In  major 


weapon  systems  design  and  production,  have  refined 
the  techniques  for  Isolated  training  tasks,  espec¬ 
ially  performance-oriented  tasks.  However,  at 
present,  there  Is  a  gap  containing  one  of  the  most 
potentially  Important  training  activities. 

Past  and  on-going  research  has  been  devo¬ 
ted  to  the  evaluation  of  the  transfer  of  training 
In  the  lower  fidelity  portions  of  the  trainer 
fidelity  spectrum  or  continuum  (e.g.,  pencil  and 
paper,  sound  slide,  etc.)  and  In  the  higher 
physical  fidelity  portions  (e.g.,  whole-mission 
or  whole-task  trainers  such  as  flight  simulators 
or  system  specific  maintenance  simulators).  For 
example,  It  Is  reported  that  the  747  simulator 
has  been  refined  to  such  a  state  that  pilots  can 
be  FAA-certlfled  in  It.  As  shown  In  Figure  4, 
however,  a  gap  in  research  exists  In  the  area  of 
low  physical-,  but  high  psychological-fidelity, 
highly  Interactive  part-task  and  part -mission 
trainers. 

The  costs  for  designing,  developing  and 
producing  small  part-task  and  part-mission  train¬ 
ers  have  come  down  considerably,  particularly  In 
the  area  of  computer-based  trainers  using  off-the- 
shelf  computers  and  delivery  device  hardware  com¬ 
ponents  with  higher  order  computer  languages.  The 
problem  Is  that  the  cost-effectiveness  of  various 
training  techniques  and  training  equipment  design 
on  transfer  of  training  has  not  been  rigorously 
validated  and  documented  well  enough  to  justify 
their  funding  In  new  systems  procurements.  Nobody 
has  yet  developed  models  for  comparisons  of  cost 
effectiveness  between  a  large-scale  simulator  and 
a  family  of  small  part-task  or  part-mission  train¬ 
ers,  or  for  comparisons  between  an  AET  and  a 
family  of  small  trainers. 

A  tri-service  coordination  of  research  and 
experimentation  In  this  area  could  assure  that  It 
is  conducted  according  to  a  master  plan  (and  a 
firm  time  table);  could  minimize  duplication  of 
effort  and  funds;  and  would  ensure  enthusiastic 
participation  by  each  of  the  three  services  in 
later  follow-on  applications. 

The  master  plan  should  Include  sub-plans 
to  investigage  the  comparative  effectiveness 
of  training  transfer  from  various  computer-based 
part-task  trainers  for  each  of  the  basic  train¬ 
ing  requirement  categories  (e.g.,  recognizing 
patterns,  making  decisions,  gross  motor  skills, 


etc.)  as  depicted  In  Table  1. 


TABLE  1 

no  .training  mmsaaB  categories 


MENTAL  SKILLS 


INFORMATION  SKILL 
PHYSICAL  SKILLS 


ATTITUOC  SKILL 


1.  RULE  LEARNING  t  USING 

2.  CLASSIFYING-RECOGNIZING  PATTERNS 

3.  IDENTIFYING  SYMBOLS 

4.  DETECTING 

s.  MAKING  oecicions 

«.  RECALLING  BOOIES  OF  INFORMATION 
7.  PERFORM  GROSS  MOTOR  SKILLS 
>■  STEER IMG/ CONTINUOUS  MOVEMENTS 
CHAINED  POSITIONING  MOVEMENTS 

10.  VOICE  CONNJNICATION 

11.  ATTITUOC 


The  training  system  requirements  are  de¬ 
rived  through  detailed  evaluation  of  each  of  the 
subtasks.  See  Figure  5.  Each  training  "system" 
to  be  evaluated  Includes  the  mini -curriculum,  the 
devices,  the  courseware,  the  software,  etc. 
designed  to  effect  transfer  of  training  for  a 
particular  part-task  or  cub-task. 
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Figure  5:  Sub-Task  Microdecomposition  Forms  The 
Basis  for  Cost-Effective  Part-Task 
T  ral ners 


The  goal  of  this  research  Into  cost  effec¬ 
tiveness  of  ptrt-task  trainers  as  compared  with 
full  system  simulators  or  AETs  wo<ild  be  'o  prove 
concepts  -  not  to  sell  hardware  or  software. 
Hardware  computational  and  delivery  devices  and 
software  "power"  are  changing  at  too  rapid  a  pace 
to  be  "spec'ed"  as  limiting  desiqn  concepts.  Only 
through  rigorous  definition  of  the  concept  to  be 
evaluated,  Its  relationship  to  the  master  plan 
hierarchy  of  lfarnlng  strategies,  and  rigorous 
cost  accounting  will  funding  authorities  for 
future  trainers  feel  confident  In  "buying  off" 
on  innovative,  cost-effective  approaches  to 
meeting  the  complex  training  challenges  of  new 
weapon  systems. 


Personnel  Subsystem  Costing  Techniques  Keyed  to 
Desiqn  Parameters  .  ~  ~ 

Until  now  we  have  discussed  techniques  for 
stretching  the  available  manpower  by  Increasing 
the  productivity  of  military  personnel.  The  other 
half  of  the  battle  lies  In  eliminating  the  re¬ 
quirements  for  many  military  personnel  at  the 
weapon  system  concept  definition  stage  by  use  of 
refined  life  cycle  cost  (LCC)  models. 


Many  LCC  models  now  exist.  Some  of  these 
consider  people  In  broad  terms  such  as  numbers, 
salary  levels,  and  skill  levels.  However,  none 
are  designed  to  provide  sensitivity  to  people- 


dependent  weapon  system  design  parameters.  In  a 
broad  sense,  they  can  reveal  gross  savings  by 
deleting  billets  from  a  proposed  TO  and  E.  How¬ 
ever,  for  a  new  weapon  system,  they  have  not  been 
proven  valid  or  reliable  enough  to  be  exercised 
rapidly  to  affect  design  decisions  at  the  concept 
definition  stage. 

In  some  cases,  meticulous  cost  trades  have 
been, developed  by  hand  to  justify  manpower  savings 
through  Innovative  configurations,  but  they  have 
been  rejected  as  outlandish.  For  example,  water¬ 
proof  carpeting  versus  tile  on  a  recent  destroyer 
design  was  shown  to  significantly  reduce  facility 
maintenance  costs  (manpower).  The  design  proposal 
was  rejected  during  concept  definition  (but  retro¬ 
fitted  ten  years  later  at  significant  cost). 
Contrarlly,  If  the  concept-definition  planners 
make  arbitrary,  unvalidated  manpower  reductions  to 
satisfy  RFP  requirements,  such  actions  reduce  the 
credibility  of  the  personnel  and  training  commun¬ 
ity. 


Thus,  LCC  models  must  be  designed  to  support 
the  weapon  system  creator  or  "synthesizer",  to 
perform  trade-offs  among  the  hardware,  software 
and  personnel  subsystems.  Furthermore,  the  models 
must  aid  the  personnel  subsystem  designer  to  in¬ 
fluence  the  hardware  and  software  design  by  showing 
the  complexity  and  cost  Impacts  of  various  proposed 
configurations  on  manning  levels  and  training 
costs.  The  models  must  be  designed  to  be  exercised 
rapidly  to  liimedlately  reflect  personnel  Impacts  of 
design  decisions,  and  permit  redesign  prior  to 
prototype  production. 

Participation  in  the  design  process,  suppor¬ 
ted  by  accurate  personnel  subsystem  LCC  data, 
permits  the  PSS  designer  to  propose  design  alter¬ 
natives  to  minimize  manpower  In  several  ways. 

These  range  from  merely  redefining  tasks  and  jobs, 
through  minor  configuration  changes,  to  total 
deletion  of  tasks  by  completely  automating  certain 
functions.  Conversely,  when  a  task  analysis  shows 
that  the  operational  or  maintenance  staff  is 
undertasked  dicing  routine  operations,  the  complex¬ 
ity  of  hardware  or  software  design  might  be  re¬ 
duced,  with  more  functions  transferred  to  the  per¬ 
sonnel  subsystem.  In  each  case,  weapon  system 
effectiveness  must  be  verified  as  being  within 
acceptable  limits. 

As  was  pointed  out  in  the  personnel  and 
training  R  and  D  factors  conference  last  May,  most 
of  the  basic  hardware  LCC  models  now  exist.  The 
problem  was  stated  as  lack  of  valid  data  and  lack 
of  skill  In  managing  or  using  the  models.  In 
development  and  use  of  the  personnel  subsystem  cost 
model,  the  data  problem  will  be  compounded-as  any¬ 
one  who  has  tried  to  find  out  true  DoD  people  costs 
well  knows.  However,  In  this  case  the  results  will 
more  than  justify  the  research. 

A  very  significant  subset  of  the  cost  model 
concerns  the  trade-offs  within  the  personnel  sub¬ 
system  development  or  training  program  itself. 

For  example,  the  perennial  controversy  of  the 
“real"  thing  (Actual  Equipment  Trainers)  versus 
simulation-based  whole  or  part  task/mission 
trainer  could  be  translated  into  objective  para¬ 
meters  for  evaluation  In  the  model.  This  would 


489 


eliminate  the  problems  caused  when  new  system 
program  managers  are  hamstrung  by  the  "color  of 
money".  It's  easier  to  buy  extra  units  of  opera¬ 
tional  equipment  (that  don't  meet  the  true 
training  need)  than  trainers.  It's  easier  to  pro¬ 
vide  (high  cost)  spares  and  support  from  the 
supply  system  (at  "training  priority"  levels). 
However,  the  results  of  the  transfer  of  training 
research  discussed  earll.er,  translated  Into 
parameters  suitable  for  trading  off  of  cost  and 
effectiveness,  would  greatly  enhance  the  credi¬ 
bility  of  the  personnel  subsystem  life  cycle 
cost  model. 

Those  who  control  the  purse  strings  for 
weapon  system  procurement  and  fielding  (O&M)  are 
basically  businessmen.  Consequently,  if  t  valid, 
reliable  and  sensitive  cost  model  can  show  them 
that  significant  personnel  subsystem  cost  savings 
over  the  life  cycle  of  the  system  can  be  achieved 
by  additional  Initial  Investments  In  the  hardware 
or  software  subsystem  of  new  or  modified  weapon 
systems,  It  would  seem  that  changing  the  "color" 
of  the  money  (08M  to  R&D  or  Procurement)  would 
not  be  a  problem.  Me  In  the  Personnel  and  Train¬ 
ing  community  have  been  talking  qualitatively 
about  such  savings  for  too  long.  We  must  now 
talk  quantitatively  in  this  Inflating  environment, 
or  we'll  never  beat  either  the  numbers  (manpower 
scarcity)  problem  or  the  cost  problem. 

Enough  talk!  We  In  the  Personnel  and 
Training  community  must  find  an  advocate  -  a 
czar  -  in  a  DoD  policy  making  position  to  Inte¬ 
grate  the  R  and  D  efforts  of  the  "four"  services 
to  produce  and  validate  such  a  model  as  soon  as 
possible.  It  must  be  designed  to  Interface  with 
existing  LCC  models  to  ensure  that  such  high  cost 
drivers  as  spare  parts  are  considered  in  the  hard¬ 
ware/software/personnel  subsystem  cost  trades. 

It  must  address  -  and  trade-off  -  both  operation¬ 
al/crew  personnel  and  logistic  support  personnel 
requirements. 

The  people  support  costs  are  possibly  the 
biggest  single  contributor  to  the  cost  per  head 
of  the  on-line  crewman  or  malntalner.  Although 
some  figures  have  been  released  recently  on  this 
cost  per  head  of  the  Individuals  assigned  to 
specific  systems  operations  and  maintenance,  the 
data  is  highly  suspect  considering  the  total 
budget  dollars  unaccounted  for.  Suffice  It  to 
say  that  each  person  eliminated  from  the  system  by 
careful  design  can  result  In  very  significant  cost 
savings  over  the  life  cycle  of  the  total  system. 
For  example,  a  factor  often  Ignored  by  both  Indus¬ 
try  and  government  In  discussing  the  "high  cost" 
of  training  for  new  weapon  systems  Is  the  fact 
that  the  student  salaries,  burdened  with  the 
costly  load  of  hidden  support  costs,  coupled  with 
their  non-productivity  during  training  far  exceeds 
the  cost  of  developing  and  conducting  the  train¬ 
ing.  In  fact,  additional  training  development 
costs  Including  those  for  developing  computer  or 
simulation  based  training,  which  shorten  training 
time,  can  often  be  more  than  offset  by  the  stu¬ 
dent  costs  saved.  These  are  the  points  that  we 
can  establish  clearly  through  the  use  of  an 
adequate  cost  model. 


Conservatism 

Five  years  ago,  the  personnel  and  training 
conmunlty  would  have  found  It  virtually  impossible 
to  obtain  widespread  military  and  Industry  appro¬ 
val  of  the  solutions  to  the  manpower  shortage 
problem  recomnended  herein.  The  stumbling  block 
would  have  been  the  conservatism  that  has  existed 
In  both  the  military  and  Industry  sectors.  The 
services  cannot  afford  to  buy  what  they  don't 
need.  Industry  cannot  afford  to  be  Innovative  In 
advancing  the  state-of-the-art  in  a  competitive 
market  place  without  Incentives. 

Now,  however.  Implementation  of  this  plan 
would  be  extremely  difficult,  but  possible.  At 
this  point,  the  military  Is  beginning  to  exoer- 
ience  the  effects  of  the  growing  manpower  short¬ 
age.  Many  high-level  policymakers  In  both  DoD 
and  Industry  understand  clearly  that  the  problem 
will  soon  grow  far  worse.  Thus,  the  plan  could  be 
Implemented  If  It  should  attract  the  full  support 
of  a  key  policymaker,  who  had  the  required  power 
over  the  necessary  resources. 

In  such  case,  the  specific  problems  resul¬ 
ting  from  conservatism  that  would  have  to  be  over¬ 
come  are: 

1)  School/user  Interface 

•  schools  are  unwilling  to  shift  some 
school  training  tasks  to  the  field 
("field  people  don't  know  how  to 
train"); 

•  field  units  are  unwilling  to  accept 
field  training  packages  prepared  by 
either  the  schools  or  contractors 
("they  don't  know  my  particular 
job"); 

•  distrust  of  self-paced  training  (you 
need  the  "soldier  role  model"); 

2)  DoD/contractor  Interface 

•  the  contractor  Is  always  trying  to 
sell  a  "bill  of  goods"  (everything 
must  be  staffed  and  managed/monl- 
tored  "to  death");  or 

3  the  customer  Is  always  trylnq  to 
"get  something  for  nothing"  ("give 
him  exactly  what  he  asks  for  whether 
it  makes  sense  or  not"); 

3)  Simulator/trainer  procurement 

•  spread  the  responsibility  by  ensuring 
that  everyone  with  a  possible  Inter¬ 
est  Is  part  of  the  proposal  evalua¬ 
tion  process; 

•  the  only  good  simulator  or  trainer  Is 
one  which  Is  "spec'd"  to  death  (re¬ 
gardless  of  whether  It.  meets  a 
training  need,  e.g.,  a  $30M  trainer 
based  on  a  $10K  "ISD"); 
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•  big  trainers  mean  big  budgets;  a 
group  of  small  part- task  or  part- 
mission  trainers  projects  are  too 
difficult  to  handle; 

4)  Joint  service  R  and  D  projects 

t  "wasn't  Invented  here  and  won't  work 
in  my  service"  preventing  effective 
cross-utilization  or  joint  sponsor¬ 
ship  of  innovative  Personnel  and 
Training  R  and  D; 

•  only  "well-published"  people  are 
qualified  to  do  R  and  D. 

CONCLUSION 


In  the  paper,  two  major  thrusts  are  recom¬ 
mended  to  ease  the  effects  of  the  looming  manpower 
shortage  upon  the  military  In  the  80 's  and  90* s: 

1)  redesign  the  training  and  selection/ 
retention  system  to  Improve  productiv¬ 
ity;  and 

2)  develop  valid  personnel  subsystem 
costing  models  to  support  the  design 
of  weapon  systems  with  less  people. 

Increased  productivity  per  Individual, 
coupled  with  reduced  weapon  system  manning 
requirements  (skills  and  numbers),  should  go  far 
In  helping  the  military  to  meet  Its  manpower 
demands  through  the  year  2000  In  spite  of  the 
projected  reduction  In  manpower  base  and  Increased 
per  capita  manpower  costs.  Although  It  Is  recog¬ 
nized  that  activities  are  underway  In  many  differ¬ 
ent  sectors  which  would  contribute  to  the  success 
of  these  efforts,  the  need  remains  to  attract  a 
strong  advocate  with  a  high  enough  authority  level 
to  establish  policy  to  Implement  an  Integrated 
manpower-conservation  program. 
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ABSTRACT 

The  Defense  Systems  Management  College  (DSMC)  was  chartered  in  1971  by  then  Deputy  Secretary  of 
Defense  David  Packard  to  be  "the  academy  of  management  for  the  Department  and  for  all  four  services." 
DSMC  has  become  a  high  priority  school  for  a  high  priority  mission:  defense  systems  acquisition  management, 
it  is  the  only  institution  that  provides  DOD  military  and  civilians  and  defense  industry  students  with  a  concen¬ 
trated  20-week  Program  Management  Course  (PMC).  A  key  feature  of  the  PMC  is  that  students  learn  not  only 
the  fundamentals  of  the  functional  disciplines  — business  management,  technical  management,  organizational 
management,  and  acquisition  policy— but  they  also  participate  actively  in  the  integration  of  these  disciplines 
through  management  case-study  simulations.  This  paper  will  describe  the  key  features  of  the  Program  Manage¬ 
ment  Course  and  some  of  the  initiatives  DSMC  is  undertaking  to  insure  its  program  meets  the  need  for  trained 
acquisition  managers  throughout  the  1980s. 


DSMC  TODAY:  EMPHASIS  ON  INTEGRATION 

The  Defense  Systems  Management  College  was  chartered 
in  1971  by  then  Deputy  Secretary  of  Defense  David  Packard  to 
be  "the  academy  of  management  for  the  Department  and  all 
four  services."  DSMC  has  become  a  high  priority  school  for  a 
high  priority  mission:  defense  systems  acquisition  manage¬ 
ment.  The  curriculum  that  has  evolved  over  the  years  has  been 
built  on  the  theme  that  learning  the  various  functional 
discip'ines  in  acquisition  is  important,  but  even  more  critical  is 
learning  how  to  integrate  these  disciplines.  It  is  absolutely 
essential  that  program  managers  (PMs)  be  master  integrators ! 
With  the  complexities  of  technology  — and  the  organizations 
involved  in  managing  defense  systems  acquisition  — program 
managers  are  faced  with  the  task  of  organizing  and  managing 
projects  that  must  interface  with  numerous  other  projects  and 
organizations.  In  order  to  achieve  balances  between  cost, 
schedule,  and  technical  performance  of  these  programs,  they 
must  be  able  to  get  their  whole  system  to  "play  together  at  the 
same  time."  It  is  not  enough  for  a  PM  to  be  a  good  engineer  or 
a  good  manager  of  people  or  even  just  a  good  businessman.  To 
succeed  in  the  world  of  program  management,  a  PM  must  be 
knowledgeable  (to  the  degree  necessary— very  difficult  to 
define)  in  many  functional  areas  and  be  exceedingly  well- 
skilled  in  the  art  of  integrating  these  functions.  The  DSMC -has 
built  its  program  around  the  idea  that  acquisition  managers 
need  experience  in  how  these  functional  areas  interface  with 
one  another  in  the  complex  arena  of  systems  acquisition. 

The  Defense  Systems  Management  College's  program  in¬ 
cludes  the  20-week  Program  Management  Course,  numerous 
short  courses  in  specific  functional  areas,  and  several 
executive-level  courses. 

The  core  of  the  DSMC  program  is  the  PMC.  It  is  attended 
by  military  and  civilian  students  from  all  services,  as  well  as 
representatives  from  defense  industry.  The  PMC  is  offered  to 
about  1 80  students  twice  each  year  and  includes  courses  in  the 
functional  areas  of  program  management:  business  manage¬ 
ment,  technical  management,  organizational  management, 
and  DOD  acquisition  policy.  Students  learn  the  fundamentals 
of  these  disciplines  in  the  functional  courses  listed  in  Figure  1. 
Each  of  the  functional  disciplines  is  complex  in  its  own  right, 
and  studying  all  of  them  at  once  compounds  the  complexity. 
What  becomes  apparent  in  teaching  program  management 
also  becomes  apparent  to  any  PM:  it  is  not  enough  to  know 
about  each  of  the  functional  disciplines;  one  must  know  how 
they  interrelate.  To  achieve  this,  the  College  puts  a  great  deal 
of  emphasis  on  integration  in  the  PMC. 


Department 

Functional  Course 

Content  Emphasis 

Organizational 
Management  and 
Policy 

Program  Management 

Data  Management-Importance  to  the 
Program  Office... The  "Data  Call" 

Business 

Management 

Contracts 

Data  Management... Requirements  and 
Description.. .Data  Rights 

Cost  Estimating 

Cost  Performance  Reports...C/SCSC... 
Budget  Data 

Technical 

Management 

System  Engineering 

Engineering  Data. ..SOW. ..Engineering 
Drawings..  Specifications.. Configuration 
Management...  Software  Data 

Logistics 

Logistic  Plan.. .Operator  Manuals  and 
Handbooks...  Parts  Manuals-Training 

Testing 

Test  and  Evaluation  Master  Plan. ..Test 
Data  and  Reports 

Figure  2.  Functional  Interrelationships:  Data  Management 


Functional  Course  Interrelationships 

The  first  step  in  teaching  integration  in  the  PMC  is  iden¬ 
tification  of  the  interrelationships  between  the  functional 
courses.  Many  areas  in  the  business,  technical,  and  organiza¬ 
tional  management  courses  relate  to  the  content  in  other 
courses.  For  example,  concepts  and  principles  of  data  manage¬ 
ment  are  covered  in  the  broad  sense  in  the  Fundamentals  of 
Program  Management  course  (Figure  2).  The  Business  Manage¬ 
ment  Department  discusses  the  contractual  aspects  of  data,  as 
well  as  the  data  requirements  in  the  world  o*  financial  manage¬ 
ment.  In  several  Technical  Management  courses,  the  faculty 
discusses  the  definition,  purpose,  common  practices,  etc.,  of 
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technical  data— drawings,  manuals,  handbooks,  plans,  and 
reports.  The  student  first  learns  these  as  separate  functions, 
but  then  must  Integrate  them  to  learn,  for  example,  how  to 
specify  data  requirements  for  a  total  program. 

Functional  Course  Caselettes 

Another  way  in  which  students  are  aided  in  understanding 
these  "real-world1'  interrelationships  in  functional  courses  is 
through  the  use  of  short  cases  or  "caselettes.”  Normally  not  ex¬ 
ceeding  one  or  two  pages  in  length,  the  caselette  becomes  a 
means  of  establishing  a  common  point  of  departure  to 
demonstrate  techniques  or  concepts  related  to  the  material  be¬ 
ing  discussed:  With  relative  ease  an  instructor  can  move  from 
discussion  with  the  class  to  a  desired  teaching  point,  using  the 
caselette  as  the  vehicle.  This  student-faculty  interaction  per¬ 
mits  learning  to  take  place  anywhere  along  the  knowledge- 
comprehension-application  learning  spectrum  (Figure  3).' 


Figure  3.  Learning-Behavior  Spectrum 


System  X 

The  third  method  used  in  the  PMC  to  achieve  integration  is 
a  series  of  management  simulation  cases  known  as  "System 
X."  System  X  ("SX,"  as  it  is  called  by  students  and  faculty)  con¬ 


sists  of  20  case  studies  with  a  common  story  line  that  takes  the 
student  from  the  beginning  to  the  end  of  a  weapon  system  ac¬ 
quisition  life  cycle  (Figure  4). 

System  X  attempts  to  simulate  the  real-world  environment 
of  acquisition  management.  Students  work  in  heterogeneous 
five-person  groups  designed  with  student  interaction  In  mind. 
Students  are  free  to  present  their  views  about  situations 
created  in  the  case-study  scenarios  and  get  involved  with  solv¬ 
ing  acquisition  problems  with  their  classmates.  Faculty 
facilitators  use  various  methods  to  have  the  students  actually 
live  the  situation  in  question.  Having  students  assume  the  roles 
of  the  characters  presented  in  the  case  or  having  them  par¬ 
ticipate  in  simple  guided  discussions  are  tne  best  ways  of  get¬ 
ting  them  to  understand  the  issues  through  such  methods.  The 
students  are  placed  In  the  position  of  having  to  actively  solve 
the  problems  they  have  discovered.  This  ability  to  think  in  new 
circumstances  is  what  the  PMC  is  all  about.  The  College 
believes  this  is  what  differentiates  the  highly  successful  pro¬ 
gram  manager  from  less  satisfactory  ones. 

Program  Management  Decision  Briefing 

A  fourth  way  in  which  a  student  can  fully  experience  the  in¬ 
tegration  of  several  disciplines  is  the  requirement  to  make  a 
program  management  decision  briefing.  This  requirement 
forces  the  student  to  select  an  Issue  from  :.n  SX  ccse  find 
prepare  a  1 5-minute  decision  briefing  (as  opposed  to  a  status 
or  information  briefing).  The  student  must  define  the  problem, 
present  alternative  solutions  with  an  analysis  of  these  alter¬ 
natives,  present  his  conclusions,  and  recommend  a  course  of 
action.  Through  this  experience,  the  student  learns  not  only 
how  to  construct  a  logical,  well-organized  briefing,  but  also 
learns  how  to  deliver  it  credibly.  At  the  conclusion  of  the  deci¬ 
sion  briefing,  a  faculty  evaluator  gives  the  student  feedback  on 
the  strengths  and  weaknesses  of  his  presentation  and  its  effec¬ 
tiveness.  This  ability  to  scope  an  issue  into  a  manageable 
15-minute  briefing  for  a  decision  adds  yet  another  dimension 
to  the  repertoire  of  skills  required  of  today's  program  manager. 

Summary:  Where  We've  Been 

DSMC's  program  was  designed  to  (1)  provide  students  with 
an  understanding  of  the  management  fundamentals  in  the 
functional  areas,  (2)  teach  them  how  those  functions  inter¬ 
relate,  and  (3)  give  them  practice  in  integrating  the  functions. 


SX  Cases 


1  2  3  4  5  67  8910  11  12  13  14  15  16  17  18  19  20 


Concept  Exploration  Phase 

Demonstration  and 
Validation 

Full-Scale  Development 

ProductionfDeployment 

Phase 

SX-1  Program  Initiation  Phase 

SX-2  Sensitivity  Analysts 

SX-3  Contingency  Analyiii 

SX-4  Acquisition  Strategy 

SX-S  System  Specification 

SX-6  Demonstration 

Validation  Decision 

SX-7  Systems  E»*:rieering 

SX-S  Design-to-Cost 

SX-S  System  Support  Concept 

SX-10  RFP  Preparation 

SX-11  Prototype  Evaluation 

SX-1 2  Preparation  lor  Source 

Selection 

SX-1 3  Contract  Negotiations 

- 

SX-14  Cost  and  Technical 

Performance  Monitoring 
SX-15  Change  Management 

SX-1S  Preparation  lor 

Production 

SX-17  Subcontractor 
Management 

SX-18  Second  Sourceflrealtout 
Decision 

SX-19  Foreign  Military  Sales 

SX-20  Support  Equipment  and 

Operational  Availability 

Figure  4.  Defense  Systems  Acquisition  Life  Cycle 
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This  initial  design  was  accomplished  in  the  early  70s  by  ex¬ 
perts  in  functional  areas  of  systems  acquisition  and  program 
management,  It  has  been  continually  updated  and  changed  as 
DOD  policy  and  management  emphases  shifted.  Over  the 
years,  new  course  directors  with  functional  expertise  have  kept 
the  content  up-to-date  by  introducing  new  lecture  material, 
student  assignments,  and  readings.  The  College  has  also  kept 
in  close  touch  with  the  systems  acquisition  community  through 
its  extensive  research  and  consulting  program.  Program 
managers  and  key  DOD  people  have  been  frequent  guest  lec¬ 
turers  at  the  College  and  bring  current  Issues  and  lersons 
learned  to  the  campus.  Thus,  the^course  content  has  been 
"quality  controlled"  extremely  well  for  the  past  10  years. 

DSMC  TOMORROW:  EMPHASIS  ON  RELEVANCE 

As  acquisition  managers  reach  into  the  1980s,  however, 
defense  systems  are  getting  more  complex  and  the  layers  in  the 
bureaucracy  continue  to  be  burdensome.  The  number  of  ac¬ 
tivities  program  managers  must  integrate  continues  to  in¬ 
crease.  As  we  look  to  the  immediate  horizon,  PMs  are  faced 
with  complexities  in  multinational  programs,  decreasing  pro¬ 
ductivity,  the  high  cost  of  energy,  long  lead  times  of  critical 
materials,  and  implementation  of  the  Carlucci  Action.,  just  to 
name  a  few.  As  the  PM's  job  becomes  more  complex,  the  Col¬ 
lege  is  initiating  several  efforts  to  insure  that  its  program  con¬ 
tinues  to  prepare  managers  to  meet  the  challenges  of  the  '80s 
(Figure  5). 


Figure  S.  Acquisition  Curriculum  Integration  Review 


Program  Management  lob  Model 

To  identify  the  baseline  of  what  the  program  manager's  job 
tasks  (as  described  by  current  PMs)  are,  the  DSMC  is  develop¬ 
ing  a  program  management  job  model.  The  effort  involves  ob¬ 
taining  a  consensus  from  a  sample  of  current,  exemplary  pro¬ 
gram  managers  on  what  they  actually  do  in  managing  their 
programs.  Six  major-system  PMs  (two  from  each  service) 
worked  together  for  a  week  and  developed  a  systems- 
engineered  model  of  their  job  functions.  The  model  was 
validated  by  several  more  PMs  from  the  Army,  Navy,  and  Air 
Force,  as  well  as  several  senior  system  commanders,  The  Col¬ 
lege  sought  to  have  working  PMs  describe  what  they  do  to  see 
how  it  compares  with  what  the  College  currently  prepares  pro¬ 
gram  managers  to  do.  Although  the  model  Is  not  in  final  format 
at  the  time  of  this  writing,  its  length  alone— 40  feet— is 
testimony  to  the  complexity  of  the  PM's  job.  It  is  no  surprise 
that  the  function  of  integration  was  emphasized  by  the  pro¬ 
gram  managers  as  being  critical  to  the  success  of  their  pro¬ 
grams! 


Senior  Executives  Evaluation 

The  College  is  also  gathering  Information  from  other 
sources  to  help  fine-tune  the  curriculum  for  the  1980s.  Senior 
management  executives  from  the  services  and  industry  are 
evaluating  the  relevance  of  course  objectives  and  content. 
They  will  be  evaluating  the  configuration  of  the  courses  and 
will  suggest  ways  to  organize  the  content  to  improve  the  effec¬ 
tiveness  and  the  integration  of  the  courses. 

PMC  Graduates 

Graduates  of  the  Program  Management  Course  will  also  be 
ranking  the  relevance  of  the  various  PMC  courses  to  their  job 
and  will  be  commenting  on  areas  they  feel  need  more  or  less 
emphasis  in  the  PMC.  This  will  help  give  a  perspective  of  how 
effective  the  PMC  is  in  preparing  students  for  job  assignments 
In  acquisition  management. 

Incoming  Students 

A  fourth  source  of  data  will  come  from  students  entering 
the  PMC.  They  come  with  varying  levels  of  experience  relative 
to  acquisition  management.  To  Identify  more  precisely  the  pro¬ 
file  of  student  knowledge,  a  pretest  inventory  of  acquisition 
management  knowledge  will  be  given  to  incoming  PMC 
students.  This  will  help  identify  how  much  the  typical  student 
already  knows,  not  only  about  the  various  functional  areas, 
but  also  how  those  areas  are  integrated.  Thus,  the  course  con¬ 
tent  can  be  better  tailored  to  the  real  education  neads  of  the 
incoming  students. 

Latest  Management  Concepts 

Another  important  source  of  data  in  determining  cur¬ 
riculum  content  is  the  latest,  most  up-to-date  management 
techniques  and  concepts.  Information  about  successful  pro¬ 
gram  managers  is  only  part  of  the  answer  to  making  a  cur¬ 
riculum  job-relevant.  The  College  must  continue  to  keep  pace 
with  the  latest  management  concepts  and  the  application  of 
new  technology  to  the  management  of  resources  and  pro¬ 
grams. 

Demographic 

While  the  College  works  to  assess  the  type  of  education 
program  needed  for  the  acquisition  management  arena,  it  will 
also  be  working  to  identify  the  potential  student  population. 
Data  will  be  obtained  from  all  the  services  on  the  number  of 
people  working  in  acquisition  management,  including  their 
functional  area,  location,  and  grade/rank.  Turnover  and  pro¬ 
motion  rates  will  be  identified  as  well  as  trends  in  skill  mixes 
and  motivators.  In  order  to  prescribe  an  educational  program 
which  will  really  meet  the  services'  needs,  the  College  is  work¬ 
ing  to  identify  what  the  acquisition  manpower  resource  will 
look  like  in  the  coming  years. 

Moving  into  the  Future 

When  the  College  has  reassessed  the  need  for  acquisition 
management  education  and  identified  the  type  of  people  who 
need  it,  the  College  will  be  in  a  position  to  analyze  ways  in 
which  its  program  might  be  improved,  DSMC  is  committed  to 
maintaining  an  education  program  which  prepares  acquisition 
managers  for  the  very  complex  job  of  integrating  and  manag¬ 
ing  programs. 

SUMMARY 

The  world  of  program  management  in  defense  systems  ac¬ 
quisition  is  complex.  The  College's  educational  program  was 
designed  to  prepare  acquisition  managers  for  this  complex  job 
by  first  providing  them  with  a  foundation  in  the  functional 
disciplines,  then  giving  them  the  opportunity  to  apply  the 
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knowledge  and  concepts  In  real-world  scenarios.  DSMC's  ob¬ 
jective  Is  to  make  its  program  even  more  of  a  lived  experienced 
bv  simulating  actual  problems  of  program  management 

It  is  likely  that  the  College  will  need  to  Increase  its  reliance 
on  simulation  in  order  to  achieve  this  objective.  It  is  difficult  to 
lecture  to  students  In  the  conventional  classroom  sense  with 
the  expectation  that  It  will  equip  them  with  the  type  of 
managerial  competencies  required  in  the  program  manage¬ 
ment  environment  of  the  '80s.  Students  must  have  the  oppor¬ 
tunity  to  apply  new  knowledge  and  practice  new  skills  at  the 
College  if  the  learning  is  to  transfer  effectively  back  to  the  job. 
Simulating  real-world  problems  that  students  can  practice,  and 
then  receive  feedback  on  their  performance,  will  cortinue  to 
be  a  key  element  in  DSMC’s  approach  to  the  future. 
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TRAINING  ASPECTS  OF  FIELDING  A  MAJOR 
WEAPON  SYSTEM  -  THE  UH-60A  BLACK  HAWK 
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TRADOC  System  Manaqer-Utl 1 1 ty  Helicopters 
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ABSTRACT 

The  Sikorsky  UH-60A  Black  Hawk  Is  the  United  States  Amy's  newest  Utility  Helicopter. 
The  Black  Hawk  was  designed  and  developed  to  replace  the  UH-1H  Iroquois  for  assault,  air 
cavalry,  and  aeromedlcal  evacuation  missions  {See  Figure  1).  The  UH-60A  carries  the 
Infantry  squcd  of  11  men  and  their  equipment  as  a  basic  load.  Particular  emphasis  during 
development  was  devoted  to  reducing  vulnerability.  Improving  crashworthiness  and  maintain¬ 
ability.  An  essential  aspect  to  fielding  of  any  weapon  system  Is  Insuring  that  all  aspucti 
of  training  on  the  new  system  have  been  considered.  The  Black  Hawk  was  Issued  to  opera¬ 
tional  units  In  May  1979  and  achieved  Initial  Operational  Capability  on  4  November  of  that 
year.  At  that  time,  all  the  resident  schools  were  open  and  functioning  and  most  parts  of 
the  "training  package"  had  been  completed.  Training  is,  of  course,  a  never  ending  process 
and  so  there  Is  still  much  to  be  done  with  the  UH-60A,  especially  In  the  areas  of  flight 
simulation,  continuation  training,  providing  training  materials  for  units  In  the  field  and 
updating  the  various  training  devices  and  literature  presently  In  use. 


INTRODUCTION 

In  1977,  the  Training  and  Doctrine  Comnand 
(TRADOC)  created  a  series  of  offices  entitled, 
"TRADOC  System  Managers  (TSM)."  The  philosophy 
behind  the  TSM  was  to  provide  the  link  between  the 
Army's  developer,  the  Development  and  Readiness 
Command  (DARCOM),  and  the  user  representative, 
TRADOC.  There  were  just  too  many  areas  receiving 
Insufficient  attention.  The  materiel  developers 
were  essentially  do<ng  an  excellent  job  within  the 
scope  of  their  responsibilities.  However,  with 
Increasing  frequency,  materiel  was  being  delivered 
to  units  In  the  field  who  were  unprepared  to 
accept  the  new  equipment.  The  situation  was  even 
more  alarming  because  of  the  large  number  of  new 
systems  that  were  to  be  fielded  In  the  late  1 970* s 
and  early  1 980' s .  The  TRADOC  System  Manager  Is 
basically  charged  with  the  responsibility  to  see 
that  TRADOC 's  missions  of  training,  personnel, 
logistics,  doctrine  and  operational  testing  have 
been  properly  accomplished  prior  to  fielding  the 
system.  Proper  doctrine  and  adequate  testing  are 
essential  elements  of  the  process  that  must  be 
carefully  thought  out  very  early  and  executed 
vigorously  to  provide  Information  for  favorable 
production  decisions.  The  areas  of  training, 
personnel  and  logistics  are  Interactive  and 
essential  to  the  success  of  a  new  system  once  It 
Is  fielded.  Of  the  three  elements,  training  Is 
the  key.  If  the  training  program  Is  Inadequate  to 
support  the  system,  the  overall  program  will 
certainly  fall.  If  training  Is  conducted  early 
and  correctly,  then  the  field  can  accept  sophis¬ 
ticated  and  highly  technical  systems  with  little 
disruption  to  normal  operations.  The  TSM  for  the 
Black  Hawk  was  appointed  in  April  1977  In  recogni¬ 
tion  of  the  scheduled  fielding  of  the  system. 

There  was  much  to  do  especially  In  the  field  of 
training. 


THE  TRAINING  PACKAGE 

This  Is  an  often  used  phrase  that  bears 
little  Identity  to  Its  true  meaning.  What  com¬ 
prises  the  package?  I  am  sure  In  defining  this 
I  will  offend  some,  If  not  all,  by  leaving  out  an 
Ingredient  or  two,  but  nonetheless,  here  is  a 
sample  of  the  major  elements  of  training  to  sup¬ 
port  new  systems  In  the  field. 

--  Training  Strategy 

--  Instructor  and  Key  Personnel  Training 

—  Resident  Training  Plans 

—  Training  Software 

--  Training  Hardware  and  Devices 

--  Flight  Simulators 

—  Plans  for  Follow-on  Training 

—  New  Equipment  Training  Teams 

Let's  take  a  look  at  each  of  these  Items  and 
hopefully  see  how  they  all  fit  together  to  Form 
the  training  package. 

Training  Strategy 

How  the  training  for  a  major  system  Is  to  be 
accomplished  is  embodied  within  the  Individual  and 
Collective  Training  Plan  (ICTP).  This  plan  Is 
developed  by  the  Director  of  Training  Developments 
assigned  to  the  TRADOC  mission  proponent  of  the 
weapon  system.  In  the  case  of  the  Black  Hawk, 
this  was  the  Infantry  School.  The  ICTP  of  the 
Black  Hawk  was  probably  the  first  such  plan 
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produced  to  support  a  major  wer  system.  The 
plan  Is  a  roadmap.  As  such,  It  .ontalns  all  the 
details;  training  concepts,  tasks  and  objective 
schedules,  logistical  support  of  training,  new 
equipment  training,  resident  courses  of  Instruc¬ 
tion,  unit  training,  Instructor  requirements, 
training  equipment  support,  training  literature, 
training  devices,  facility  requirements,  and 
probably  most  Important,  the  requirement  for 
resources  and  funding.  The  ICTP  encompasses  the 
training  plans  for  all  those  locations  where 
training  takes  place.  For  the  Black  Hawk,  It  covers 
the  comprehensive  plans  of  Fort  Eustls,  VA,  for 
maintenance,  Fort  Rucker,  AL,  for  pilot  training, 
Fort  Gordon,  GA,  for  avionics  training  and  the 
Armor  and  Infantry  Schools  and  the  Academy  of 
Health  Sciences  for  their  particular  Interest  In 
the  doctrinal  use  of  the  Black  Hawk.  The  ICTP  was 
approved  for  TRAOOC  by  the  TSM  and  forwarded  to 
the  Denartment  of  the  Army.  It  Is  probably  the 
first  document  that  DA  receives  that  Identities 
what  must  take  place  In  order  to  accommodate  a  new 
system.  The  ICTP  Is  without  a  doubt  the  most 
Important  single  document  Involved  In  the  estab¬ 
lishment  of  the  training  process  for  a  new  system. 
Admittedly,  It  rapidly  becomes  dated  as  production 
schedules,  funding,  and  other  priorities  change, 
but  as  mentioned,  It  Is  a  superb  reference  document 
for  what  must  be  done. 

Instructor  and  Key  Personnel  Training  ( I  KPT ) 

This  training  is  set  up  and  funded  by  the 
DARCOM  Project  Manager.  The  training  Involves 
exactly  what  Its  name  Implies  and  Is  normally  con¬ 
ducted  by  the  contractor.  Ideally,  you  would  want 
the  contractor  to  give  Instruction  to  potential 
instructors  In  the  same  manner  that  you  would 
expect  students  In  the  future  to  receive  It.  To 
do  this,  the  Project  Manager  funded  an  extensive 
effort  with  the  contractor  to  develop  the  program 
of  instruction  for  the  I  KPT  to  represent  what 
should  be  taught  In  the  Army's  resident  schools. 
Some  $800,000  was  allocated  for  this  effort.  In 
essence,  the  contractor  performed  the  front  end 
analysis  and  the  definition  of  task  objectives  so 
vital  to  creating  a  program  of  Instruction.  Aqaln, 
this  was  new  territory.  At  that  time,  the  Army 
Initiated  Integrated  Technical  Documentation  and 
Training  (ITDT)  now  known  as  Skill  Performance 
Aids  (SPAs).  However,  the  Black  Hawk  was  developed 
prior  to  initiation  of  this  Innovative  process  of 
training  and  could  only  catch  a  portion  of  It. 

The  key  point  here  Is  that  there  must  be  a  con¬ 
certed  cooperative  effort  between  the  Project 
Manager,  contractors  and  the  TRADOC  schools  In 
establishing  this  entire  process;  If  done  In  Isola¬ 
tion  by  any  major  command,  the  tralnlnq  set  up  will 
fall. 

Resident  Training  Plans 

The  three  schools  that  train  skills  In  the 
maintenance  and  operation  of  the  Black  Hawk  are: 
the  Aviation  School  at  Fort  Rucker,  the  Transporta¬ 
tion  School  at  Fort  Eustls,  and  the  Signal  School 
at  Fort  Gordon.  Many  other  Installations  teach 
subjects  Involving  the  Black  Hawk,  but  more  In  the 
operational  use  as  opposed  to  training  particular 
skills.  In  support  of  the  Government  Competitive 
Test,  the  Army  had  to  train  individuals  In  the 
operation  and  maintenance  of  the  aircraft.  How 
much  training  and  what  to  train  was  a  constant 
subject  of  debate  when  the  Army  decided  to  provide 


the  p  lots  with  nine  hours  of  actual  operation  of 
the  aircraft  and  a  ground  school  of  some  SO  hours 
in  length.  Enlisted  mechanics  were  provided  con¬ 
tractor  courses  of  varying  lengths  depending  upon 
their  particular  Military  Occupational  Specialty 
(MOS).  Since  then  the  courses  have  evolved  as 
shown: 


COURSE/MOS  DURATION 


Aviators  and  Instructor  Pilots 

4 

weeks 

Tactical  Transport  Helicopter 

Repairer  —  *67T10/20 

8 

wks/1  day 

Senior  Tactical  Transport  Heli¬ 
copter  Repairer  --  *67T30 

0 

wks/3  days 

Aircraft  Power  Plant  —  68B 

3 

wks/3  days 

Aircmt  Power  train  Repairer  —  68D 

** 

si 

wks/2  days 

Aircraft  Electrician  —  68F 

6 

wks/1  day 

Aircraft  Structural  Repairer  — 68G 

1 

wk/3  days 

Aircraft  Pneudrcullcs  Repairer  — 

68H 

3 

wks/1  day 

Maintenance  Test  Pilot  Course 

3 

wks/4  days 

♦New  Military  Occupational  Specialty 
(See  Figures  2  and  3) 


This  brief  discussion  of  resident  training 
plans  would  not  be  complete  without  mentioning 
development  of  training  requirements.  Training 
requirements  are  developed  by  Department  of  the 
Army  and  passed  to  TRADOC  for  preparation  and 
resourcing  the  Courses  of  Instruction.  The  process 
by  which  this  Is  done  Is  Incredibly  complex  and 
out  of  synchronization  with  the  materiel  develop¬ 
ment  process.  For  example,  the  process  starts  with 
the  development  of  a  TOE  at  the  proponent  school 
which  Is  forwarded  through  TRADOC  to  DA  for  ap¬ 
proval  and  eventually  distributed  to  the  field. 

The  field  then  modifies  this  document  and  the 
result  Is  then  fed  to  a  group  of  computer  models 
among  which  are  VTAADS,  TAADS,  SACS,  and  PERSAC. 
From  this,  the  Military  Personnel  Center  (MILPERCEhO 
can  conduct  a  Personnel  Inventory  Analysis  which 
examines  such  things  as  how  many  UH-1  mechanics 
should  we  train  In  the  Black  Hawk  or  what  should 
we  do  with  the  displaced  skills  of  the  CH-54 
trained  mechanics  since  the  aircraft  has  been 
assigned  to  the  National  Guard  or  yet  what  Informa¬ 
tion  should  be  given  to  the  Recruiting  Command 
concerning  new  accessions.  The  result  Is  the 
determination  of  training  requirements  for  which 
TRADOC  develops  courses  of  Instruction  which  in 
turn  are  announced  by  the  Department  of  the  Army. 
This  process  In  the  most  optimistic  terms  takes 
35  to  40  months.  The  situation  becomes  even  more 
tenuous  (as  It  did  in  the  case  of  the  Black  Hawk) 
if  you  accelerate  the  development  cycle.  The 
Black  Hawk  Project  Manager  appeared  before  the 
Defense  Systems  Acquisition  Review  Council  (DSARC) 
In  November  1976  seeking  a  Low  Rate  Initial  Pro¬ 
duction  (LRIP)  decision  and  was  told  to  enter  full 
production  immediately.  That  single  decision 
removed  18  months  from  the  development  cycle, 
making  It  very  difficult  for  the  training  community 
to  keep  pace.  In  spite  of  this,  UH-60A  met  all 
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training  coronltments  but  only  by  expediting  and  In 
some  cases  circumventing  established  procedures. 

Training  Software 

One  of  the  training  factors  receiving  too 
little  emphasis  was  training  software.  While  soft¬ 
ware  because  of  Its  very  connotation  may  riot  seem 
Important,  much  of  the  UH-60A  training  software 
was  key  to  the  success  of  the  program.  At  the 
heart  of  this  subject  are  the  publications  to 
support  the  system.  It  not  only  allows  the 
Instructors  In  the  schools  to  conduct  their 
classes,  hut  It  allows  the  most  Important  element, 
the  trainer  In  the  field,  to  conduct  Individual 
and  unit  tralninq.  Software,  In  addition  to  the 
manuals  on  the  aircraft.  Included  training  plans 
for  resident  Instruction  and  new  equipment  training 
extension  courses.  The  preparation  for  this  type 
of  training  In  turn  develops  a  demand  for  course 
outlines,  lesson  plans,  student  Information  sheets, 
handouts,  audio  visual  aids,  practical  exercises, 
student  worksheets,  examinations  and  evaluation 
forms. 

A  certain  amount  of  doctrinal  literature  was 
also  required.  This  was  used  more  to  educate 
leaders  and  operators  In  the  capabilities  In 
employment  of  the  Black  Hawk.  These  manuals  cover 
performance  data,  load  planning  data  for  both 
troops  and  equipment,  safety  Information,  and  the 
details  of  the  tactical  employment  of  air  mobile 
and  parachute  forces.  The  air  crew  training 
manuals,  probably  more  than  anything  else,  typifies 
the  software  necessary  to  support  a  weapon  system. 
This  particular  manual  specifies  in  some  detail 
the  tasks  an  aviator  must  accomplish  to  achieve  the 
required  level  of  training  to  complete  his  combat 
mission.  After  performing  to  the  requirements  of 
the  air  crew  training  manuals,  skill  qualification 
tests,  periodic  unit  checkrldes,  and  visits  by  the 
Director  of  Evaluation  and  Standardization  at 
Fort  Rucker  are  used  to  evaluate  the  training 
level  of  pilots  and  mechanics.  The  Army  Training 
Evaluation  Program  is  used  to  evaluate  the  status 
of  unit  training  readiness.  However,  the  core  to 
this  process  remains  training  software. 

Training  Hardware  and  Devices 

Closely  tied  to  the  development  of  the  Black 
Hawk  was  the  development  of  the  associated  training 
devices  which  In  some  cases  are  almost  as  complex 
as  the  aircraft.  Taking  the  lead  In  this  area  Is 
the  DARCOM  Project  Manager.  The  Project  Manager 
requested  that  the  prime  contractors  for  the  air¬ 
craft  and  the  engine  provide  reports  recommending 
specific  training  devices  that  could  be  made 
available.  Then  each  school  responsible  for 
Black  Hawk  training  within  TRADOC  was  tasked  to 
identify  their  particular  training  device  require¬ 
ments  to  support  courses  of  instruction.  The 
requirement  for  training  devices  was  then  staffed 
through  Headquarters  TRADOC, Included  In  the  ICTP 
and  the  device  development  program  was  then  funded 
and  managed  by  the  DARCOM  Project  Manager. 

The  devices  fall  Into  three  categories: 

(1)  part  task  trainers,  (2)  composite  trainers, 
and  (3)  the  aircraft  and  its  components  (see 
Figures  4,  5  and  6).  This  effort  represented  one 
of  the  Army's  first  attempts  to  support  a  major 
weapon  system  with  a  complete  package  of  training 
devices  at  the  training  location  prior  to  arrival 


of  students.  Ms  ted  here  are  those  devices 
required  by  the  resident  schools. 

Transportation  School 
Composite  Trainer 
Power  Plant/Drive  Trainer 
Hydraulic  System  Trainer 
Powertrain  System  Trainer 
Electrical  System  Panel  Trainer 
Electrical  System  Programmable  Trainer 
Caution/ Advisory  Panel  Trainer 

Signal  School 

Stability  Augmentation  Traine“ 

Command  Instrument  System  Trainer 

Aviation  School 
Prototype  Aircraft 
T-700  Engine  Simulator 
Fuel  System  Panel  Trainer 
Hydraulic  Panel  Trainer 
Electrical  System  Panel  Trainer 
Fire  Detection  System  Trainer 
Caution/ Advisory  Panel  Trainer 
Command  Instrument  System  Trainer 
Horizontal  Situation  Indicator  Trainer 
Doppler  Navigation  System  Trainer 
UH-60A  Flight  Simulator 

Essential  to  the  success  of  the  training 
device  program  was  the  Involvement  of  TRADOC 
schools  and  their  instructors  during  the  various 
progress  reviews.  To  Insure  user  satisfaction, 
the  user  did  and  always  should  participate  in  the 
critical  design  reviews  and  the  decisions  leading  to 
the  acceptance  of  the  training  devices.  After  all, 
they  have  to  live  with  them  and  use  them  on  a 
daily  basis. 

The  training  devices  were  completed  and 
delivered  close  to  schedule.  The  major  problem 
with  the  training  device  is  like  the  major  system 
itself.  At  some  point,  the  design  must  be  frozen. 
If  this  is  not  done,  the  contractor  continually 
changes  and  updates  rather  than  producing.  If  the 
training  device  is  sufficient  to  train  the  opera¬ 
tion  of  the  system,  then  it  should  be  fielded  and 
an  update  accomplished  later. 

Flight  Simulator 

A  UH-60  flight  simulator  is  under  development 
to  compliment  aviator  training.  Use  of  the  flight 
simulator  is  planned  for  initial  aircraft 
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qualification  training  as  well  as  continuation 
training.  The  prototype  Black  Hawk  simulators  are 
presently  undergoing  an  operational  test  at  Fort 
Rucker  and  scheduled  for  completion  In  1982. 

The  relatively  high  cost  of  the  aircraft, 
fuel,  maintenance  and  facilities  dictate  that 
lower  cost  training  methods  be  used  where  possible. 
Flight  simulators  have  been  widely  accepted  as 
cost  effective  alternatives  or  complements  to 
actual  flight  time  In  aircraft.  While  cost  savings 
with  the  UH-60  flight  simulator  are  only  forecast's 
at  this  time,  actual  savings  will  be  determined 
when  the  results  of  the  operational  test  have  been 
properly  analyzed.  Cost  savings  are  dependent  on 
the  amount  of  simulator  time  that  can  be  sub¬ 
stituted  for  actual  aircraft  training  time, 
referred  to  as  the  transfer  ratio. 

While  transfer  ratios  for  qualification  and 
continuation  training  will  quantify  cost  savings, 
there  are  other  important  benefits  that  are  very 
difficult  to  quantify.  Examples  of  these  more 
abstract  benefits  Include  the  following:  up  to 
200  emergency  procedures  can  be  practiced  In  the 
Black  Hawk  simulator,  many  of  which  cannot  be 
done  In  the  aircraft.  Training  can  be  conducted 
any  time  and  is  not  dependent  on  light  and  weather. 
In  fact,  light  and  weather  can  be  simulated  as 
required  for  training,  and  maximum  safety  is 
provided  for  the  aviators.  A  mission  or  maneuver 
can  be  recorded,  frozen,  restarted,  played  back  in 
real  or  slow  time,  and  retrieved  for  debriefing. 
Since  maneuvers  can  be  simply  repeated,  the  need 
to  go  around,  land  and  perform  other  fuel  consuming 
actions  is  eliminated.  The  result  is  more  effi¬ 
cient  use  of  training  time  and  a  better  trained 
aviator. 

An  important  operational  lesson  was  learned  in 
developing  the  UH-60  Flight  Simulator.  While  it 
Is  highly  desirable  to  conduct  parallel  develop¬ 
ment  and  fielding  of  an  aircraft  and  its  companion 
flight  simulator,  it  Is  virtually  impossible  to 
final  stage  a  simulator  while  the  basic  aircraft  is 
in  a  competitive  prototype  stage.  Far  too  many 
design  changes  occur  between  a  prototype  aircraft 
and  the  production  model,  and  the  last  thing  the 
aircraft  manufacturer  designs  is  the  cockpit. 
Conversely,  the  cockpit  is  the  first  thing  the 
simulator  manufacturer  designs.  As  a  consequence, 
simulator  manufacturers  must  wait  until  the  air¬ 
craft  design  freezes  before  completing  the 
simulator  design.  The  end  result  is  that  simulator 
availability  falls  well  behind  fielding  of  the 
aircraft. 

With  the  Black  Hawk,  simulator  procurement 
has  been  further  complicated  by  the  fact  that  we 
are  competitively  evaluating  two  very  different 
visual  display  systems.  One  system  is  a  relatively 
conventional  camera  model  system  which  uses  an 
elaborate  terrain  board.  The  camera  model  system 
produces  a  very  high  resolution  visual  display. 

The  other  system  uses  digital  image  generation  and 
is  totally  computer  controlled.  While  the  computer 
generated  imagery  has  less  resolution  and  terrain 
detail  than  the  camera  model  system,  it  requires 
much  less  space  and  power  (See  Figure  7). 

As  mentioned  previously,  the  Black  Hawk 
simulator  operational  test  is  now  being  conducted. 
The  test  started  in  April  of  this  year  and  is  now 
scheduled  for  completion  by  mid-year  1982.  The 


operational  test  will  be  the  basis  for  selection 
of  either  the  camera  model  system  or  the  digital 
image  generation  system.  It  will  also  provide 
data  to  develop  the  appropriate  training  transfer 
ratios. 

The  operational  test  is  being  conducted  in 
two  stages.  The  first  stage  is  the  institutional 
portion  and  uses  actual  student  pilots  who  are 
being  qualified  in  the  Black  Hawk  at  the  Aviation 
School.  The  second  stage  will  use  pilots  already 
qualified  in  the  Black  Hawk  to  determine  the 
transfer  ratios  for  continuation  training. 

Plans  for  Follow-On  Training 

One  form  of  follow-on  training  previously 
mentioned  was  continuation  training.  The  continua¬ 
tion  training  in  units  with  Black  Hawks  takes 
several  forms,  but  they  are  all  designed  to  result 
in  a  unit  trained  to  be  mission  ready.  The  pilots 
continue  to  train  using  the  Aircrew  Training 
Manuals  (ATM).  The  enlisted  mechanics,  supervisors 
and  component  repairmen  continue  their  in-unit 
training  under  the  Skill  Qualification  Test  (SQT) 
program.  Units  will  also  be  evaluated  under  the 
Army  Training  Evaluation  Program  (ARTEP). 

Resident  training  at  the  various  schools  will 
continue  to  train  pilots,  mechanics  and  component 
repairmen.  The  resident  courses  will  be  period¬ 
ically  reviewed  for  content  and  effectiveness  and 
be  updated  as  required.  Some  updating  will  be 
required  due  to  the  different  type  students 
anticipated  in  the  future.  Most  of  the  Black  Hawk 
crew  chiefs  and  mechanics  were  previously  trained 
as  UH-1  mechanics.  Their  Black  Hawk  training  was 
in  the  form  of  a  transition  to  a  different  aircraft. 
In  the  future,  there  will  be  a  shift  to  initial 
training  on  the  Black  Hawk  direct  from  basic 
training.  This  program  has  alread.'  started 
and  will  increase  as  more  aircraft  are 
fielded. 

The  component  repairmen  have  similarly 
received  their  Black  Hawk  training  on  a  separate 
qualification  basis.  In  the  future,  all  component 
repairmen  will  receive  Black  Hawk  training. 

Since  the  UH-60A  replaces  the  UH-1  and 
literally  all  Army  helicopter  pilots  are  rated  in 
the  UH-1,  the  UH-60A  qualification  program  included 
only  pilots  already  rated  in  utility  aircraft.  In 
the  future,  we  may  see  a  separate  UH-60A  track  in 
Initial  Entry  Rotary  Wing  Aviator  training.  When 
this  occurs,  a  more  extensive  black  Hawk  training 
program  will  be  required  to  replace  the  utility 
training  formerly  taught  in  the  UH-1. 

Exportable  training  packages  are  another  form 
of  follow-on  training.  An  example  is  door  gunner 
training.  This  is  conducted  in  the  unit  with  the 
aid  of  an  exportable  training  package.  A  Black 
Hawk  pilot  exportable  training  package  is  being 
developed  for  possible  field  qualification  of 
pilots  in  the  future.  While  this  package  is  not 
yet  certified  to  qualify  an  aviator  in  the  unit,  it 
is  presently  used  for  refresher  training.  Already 
developed  and  in  the  units  are  six  TEC  lessons,  21 
ETV  tapes  and  a  series  of  lesson  plans  and  slides. 

New  Equipment  Training  Teams 

In  addition  to  the  resident  training  courses, 
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New  Equipment  Training  Teams  (NETT)  have  been  used 
to  provide  training  In  the  units  receiving  Black 
Hawks.  The  NETT  normally  accompanies  the  Initial 
fielding  and  provides  training  for  the  enlisted 
supervisors  at  the  Installation.  Composition  of 
the  teams  have  varied  from  a  low  of  three  Instruc¬ 
tors  to  the  projected  15  instructors  to  be  used  In 
the  Europe  fielding  effort. 

New  Equipment  Training  Is  designed  to  be 
given  one  time  at  each  Installation  receiving 
Black  Hawks.  When  the  NETT  effort  Is  complete,  a 
training  package  Is  left  with  the  unit.  This 
package  Includes  lesson  plans  and  appropriate 
audio  visual  aids  to  support  the  conduct  of  follow- 
on  courses  as  required. 

SUFWARY 

What  has  been  the  result  of  all  this?  It 
may  be  too  early  to  tell,  but  at  least  on  the 
surface.  It  appears  that  the  job  of  training 
operators  and  malntalners  for  the  Black  Hawk  Is 
well  In  hand.  It  was  not  easy  and  not  without 
many  simple  mistakes  committed  along  this  critical 
path.  The  aircraft  has  been  In  the  field  for 
almost  30  months,  and  there  have  been  very  few 
Incidents  and  none  directly  attributable  to  lack 
of  or  Insufficient  training.  I  guess  this  Is 
proof  enough. 

ABOUT  THE  AUTHOR 

Colonel  Cornelius  F.  McGIlllcuddy,  Jr. 

TRADOC  System  Manager-Utility  Helicopters 
US  Army  Aviation  Center 
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—Served  as  Commander  of  an  Assault  Hel  icopter 
Company  and  Combat  Aviation  Battalion  In 
Combat  In  the  Republic  of  Viet  Nam 
—Attended  Naval  War  College  in  1970-1971 
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THE  UH-60A,  BLACK  HAWK 


HANDS-ON  TRAINING  WITH  ACTUAL 
AIRCRAFT  COMPONENTS 
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ABSTRACT 


The  proof  of  combat  readiness  has  been,  and  remains,  the  demonstrated  ability  of 
a  tactical  unit  to  move,  shoot  and  communicate.  Modern  technology  Is  providing  the 
Army  with  weapon  systems  that  have  stabilized  firing  platforms  (so  as  to  shoot  on  the 
move)  and  the  capability  to  engage  targets  day  or  night  In  all  weather  conditions. 
Crews  of  these  modern  weapon  systems  should  train  on  ranges  that  challenge  them  to 
Identify  and  decisively  engage  the  dominant  threat  vehicle  from  a  multiple  array. 

Proof  of  crew  proficiency  Is  a  hit  on  the  crucial  target  by  the  correct  threat 
defeating  munition.  Training,  however,  will  not  be  enhanced  until  the  U.S.  Army 
replaces  current  nondescript  two  dimensional  plywood  targets.  Immobile  U.S.  tank 
hulks  and  anti-aircraft  targets  that  are  Incapable  of  providing  realism  and  threat 
vehicle  identification  with  targets  that  provide  realistic  size,  shape  and  thermal 
characteristics. 


The  phrase  "The  Army  must  train  as  It  will 
fight"  is  as  valid  today  as  It  was  for  the  Roman 
Legions  centuries  ago.  To  the  Legions  training 
was  a  "bloodless  war"  and  war  but  "bloody  train¬ 
ing." 

Realistic  training  today,  however.  Is  neither 
as  simple  nor  as  cheap  as  It  was  for  the  Romans. 
Close  order  hand  to  hand  combat  with  swords  and 
spears  has  given  way  to  ever  Increasing  dispersion 
and  engagements  that  begin  beyond  the  horizon  with 
weaponry  whose  munitions  are  as  destructive  as  they 
are  swift. 

It  is  common  knowledge  that  modern  weapon 
systems  possess  multiple  munition  selections, 
stabilized  weapon  platforms  and  all  weather  night 
and  day  engagement  potential.  What  isn't  as  well 
known  is  the  adverse  impact  these  advances  in 
technology  have  had  on  existing  equipment  and 
facilities. 

Today  a  gunner  must  not  only  demonstrate  his 
ability  to  hit  a  target,  he  must  also  confirm  his 
capability  to  select  the  proper  threat  defeating 
munition  for  the  target  engaged.  Additionally  if 
we  are  to  train  as  we  will  fight,  our  weapon  crews 
must  also  be  able  to  identify  and  decisively 
engage  the  dominant  threat  target  from  a  multiple 
array  of  targets. 

It  is  essential,  therefore,  that  current  and 
future  gunner  and  crew  training  be  conducted  on 
ranges  that  possess  targets  that  provide  realistic 
size,  shape  and  thermal/radar  reflective  character¬ 
istics.  These  requirements  sound  the  death  knoll 
for  single,  non-descript,  two-dimensional  plywood 
targets  and  immobile  old  tank  hulks! 


In  an  effort  to  define  and  articulate  Army 
target  requirements,  TRADOC  hosted  a  "Targets  for 
the  Eighties"  Workshop  during  January  1981.  The 
results  of  that  meeting  are  provided  below.  Hope¬ 
fully,  by  having  identified  what  is  needed  "down 
range,"  the  mechanisms  to  lift  and  move  the  various 
targets  can  be  fully  developed  and  standardized. 

As  proponent  for  Training  Circular  25-2, 
Training  Ranges,  the  Deputy  Chief  of  Staff  for 
Training  (DCST),  TRADOC,  is  keenly  aware  of  current 
U.S.  Army  range  equipment  limitations.  He  also 
realizes  that  many  commands  desire  to  procure 
commercial  range  equipment  in  an  effort  to  enhance 
gunnery  training  and  fully  appreciates  the  reasons 
for  this.  However,  the  continued  uncontrolled 
procurement  of  commercial  systems,  will  not  only 
become  a  costly  maintenance  liability  to  the  U.S. 
Army,  but  it  will  adversely  affect  the  standardi¬ 
zation  of  targets,  ranges  and  training. 

Light  Targets 

9  Definition.  Light  targets  represent  individual 
soldiers  and  multiple  silhouettes  (e.g.,  command 
group,  machinegun/ATGM  crew).  These  targets  will 
be  used  in  basic  rifle  marksmanship,  crew-served 
weapons  training,  and  on  unit  collective  training 
ranges. 

•  Characteristics. 

a.  Provide  a  three  dimensional  bas-relief 
view  in  order  to  provide  realism,  target  identi¬ 
fication,  and  engagement  priority  training. 
(RATIONALE:  A  three  dimensional  targets  aids  in 
target  identification  and  allows  interlocking 
fields  of  fire  (mutually  supporting  fire). 
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b.  Must  be  Identifiable  at  a  minimum  distance 
of  500  meters. 

(RATIONALE:  By  presenting  the  same  visibility  as 
a  man  In  the  open  to  a  distance  of  500  meters, 
soldiers  can  acquire/sharpen  detection  skills). 

c.  Size  and  color  must  as  a  minimum  represent 
Warsaw  Pact  Individuals  and  crews. 

d.  Must  be  capable  of  withstanding  multiple 
hits  from  non-fragmenting  munitions  (up  to  25-mm), 

e.  Must  be  capable  of  sensing  hits  and  kills 
from  MILES  and  .22  caliber  thru  25-mm  munitions 
(may  be  an  add-on  device).  * 

f.  Must  be  durable  and  capable  of  withstanding 
climatic  and  weather  variances  world-wide. 

g.  Must  be  Identifiable  beyond  500  meters  by 
soldi ers/crews  using  visual  aids  and/or  gun 
sights. 


h.  Provide  realistic  thermal /Infrared  sign¬ 
ature  for  Identification  during  periods  of  re¬ 
duced  visibility  (may  be  an  add-on  device).  * 

1.  Must  be  easily  repaired  at  field  site  and 
maintenance  requirements  must  be  simple. 

j.  Must  be  capable  of  being  mounted  and  oper¬ 
ating  on  stationary  and  moving  target  elevating 
mechanisms. 

k.  Must  at  a  minimum  represent:  Individual 
soldier  (standing  and  prone);  command  group; 
machlnegun  crew;  ATOM  crew;  antitank  gun  crew. 

l.  Must  be  rigid  enough  to  be  moved  on  a 
target  moving  device  at  variable  speeds  of  0-8 
mph. 

Heavy  Targets 

e  Definition.  Heavy  targets  represent  threat 
ground  vehicles  or  hovering  aircraft.  These 
targets  will  be  used  In  basic  gunnery  and  unit 
collective  training. 

e  Characteristics. 

a.  Provide  a  three  dimensional  bas-relief  view 
In  order  to  provide  realism,  target  identification, 
and  engagement  priority  training. 

b.  Must  be  identifiable  within  2-km  of  obser¬ 
vation  point. 

c.  Size,  color  and  markings  must,  as  a  minimum 
duplicate  Warsaw  Pact  vehicles.  The  capability 

to  replicate  North  Korean  or  other  nations  vehi¬ 
cles  is  desirable. 

d.  Must  be  capable  of  withstanding  multiple 
hits  (all  targets). 

e.  Aircraft  targets  must  be  rigid  enough  to 
explode  the  STINGER/REDEYE  warhead. 

f.  Must  be  capable  of  sensing  hits  and  kills 
from  MILES  and  all  munitions  .22  caliber  and  above 
(may  be  an  add-on  device).  * 


(RATIONALE:  Miles  and  .22  caliber  to  .50  caliber 
are  used  on  scaled  targets.  Miles  and  .50  caliber 
and  above  are  used  for  "full -up"  training). 

g.  Must  be  durable  and  capable  of  withstanding 
climate  and  weather  variances  world-wide. 

h.  Must  be  easily  repaired  at  field  site  and 
maintenance  requirements  must  be  simple. 

1.  Must  be  capable  of  being  mounted  and  oper¬ 
ated  on  stationary  and  moving  target  elevating 
mechanisms. 

j.  Must  provide  realistic  simulation  of  gun/ 
missile  firing  (may  be  an  add-on  device).  * 

k.  Must  provide  realistic  simulation  (smoke, 
flash,  flame,  etc.)  when  hit  (may  be  an  add-on 
device).  * 


l.  Must  provide  LASER  reflectivity  (may  be  an 
add-on  device).  * 

m.  Must  provide  thermal,  Infrared  and  milli¬ 
meter  wave  detection  signature  (may  be  add-on 
devices).  * 

n.  Aircraft  targets  must  have  capability  of 
providing  an  Infrared/radar  reflection  (may  be 
add-on  devices).  * 

o.  Aircraft  targets  must  be  In  attack  profile. 

p.  Must  be  rigid  enough  to  be  moved  on  a 
target  moving  device  at  variable  speeds  of  0-25 
mph. 


q.  Vehicle  targets  must  be  available  in  full 
size  and  1:60;  1:30;  1:10;  1:5  scales  for  various 
gunnery  applications. 

r.  Must  at  a  minimum,  represent  the  following 
Warsaw  Pact  equipment: 

ZSU  23-4  anti-aircraft  gun  (front  and 

flank) 

T-62  tank  (front  and  flank) 

T-72  tank  (front  and  flank) 

PT-76  amphibious  tank  (front  and  flank) 
BRDM  scout  car  (front  and  flank) 

BMP  armored  personnel  carrier  (front  and 

flank) 

GAZ-66  track  (flank) 

Mi-24  (HIND-Du  helicopter)  (front) 

M-35  (122-mm  gun)  (flank) 


*  Requires  effective  interface  with  target 
elevating  mechanism  and  other  range  control 
equipment. 
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TARGET  CHARACTERISTIC  REQUIREMENTS  MATRIX 


LIGHT  TARGETS 


AIR  DEFENSE  ARMOR  ARTY  AVN 


ENGR  INF 


3D  X 

Visual  Identification  X 

Hit  Sensing  -  Munitions  X 

Capability  -  Laser  Beam  X 

Fixed  Route  X 

Variable  Route 
Thermal  Imagery 

Size  (NATO  Standard)  X 


X 

X 

X 

X 

X 


X 

X 

X 

X 

X 


X  X  X 

X  X  X 

XXX 
XXX 
XXX 


X  XXX 

X  X  X  X  X 


HEAVY  TARGETS 


3D  for  Targets  within  2  HI  X 

Visual  Identification  X 

Hit  Sensing  Capability  X 

-  Munitions  X 

•  Laser  Beam  X 

Size  SCALED 

Fixed  Route  (Ground  Targets)  X 

Variable  Route  (Ground/Air)  X 

Infrared  Signature  (IR)  X 

Radar  Reflective  (RF)  X 

Thermal  Signature 

IR  Counter  Measure  (IRCM)  X 


X 

X 

X 

X 

X 

NATO 

X 


X 


X 

X 

X 

X 

X 

FULL 

X 

X 


X 


X 

X 

X 

X 

X 

FULL 

X 

X 


X 


X 

X 

X 

X 

X 

FULL 

X 


X 

X 

X 

X 

X 

FULL 

X 


X 


("X"  INDICATES  AN  IDENTIFIED  REQUIREMENT) 


SUMMARY 


The  recently  published  FM  71-999A  (DRAFT),  Infantry  and  Cavalry  Fighting  Vehicle 
Gunnery,  states:  "To  capitalize  on  the  benefits  derived  from  firing  your  weapon  first, 
the  whole  Fighting  Vehicle  crew  must  be  highly  competent  In:  detecting  the  targets; 
locating  targets;  Identifying  targets  as  friendly  or  enemy;  and  classifying  targets  as 
most  dangerous,  dangerous,  or  least  dangerous." 

The  U.S.  Army  has  equipment  second  to  none.  Shouldn't  It  also  have  training  second 
to  none?  It  will  not  until  plywood  and  junk  Is  replaced  by  targets  that  are  realistic. 
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ABSTRACT 

Currently,  the  Department  of  Defense  is  procuring  the  most  sophisticated  weapons 
systems  in  the  history  of  this  country.  Unfortunately,  this  is  resulting  in  a  rather 
serious  problem  in  that  current  technological  developments  are  vastly  outpacing  educational 
systems  approaches.  Studies  reveal  that  a  wide  gap  is  becoming  evident  between  the  skills 
possessed  by  the  high  school  graduates  and  their  ability  to  deal  with  complex  weapons 
systems.  From  this,  it  can  be  deduced  that  increased  emphasis  must  be  placed  upon  the 
design  and  development  of  educational  and  training  methodologies  for  these  systems.  In¬ 
novative  and  motivating  training  data,  which  includes  everything  from  simple  handbooks  to 
complicated  scenarios,  must  be  provided  by  the  contractors  concurrently  with  the  piece  of 
equipment.  In  addition,  this  data  must  meet  the  needs  of  the  target  audience  in  the 
military  today. 

Ideally,  these  materials  are  to  be  designed  in  compliance  with  the  Instructional 
Systems  Development  (ISD)  model  and  must  meet  specifications  required  by  Data  Item  Descrip¬ 
tions  (Dios)  which  vary  from  contract  to  contract.  In  addition,  whenever  possible  these 
materials  should  be  performance-based  and  criterion  referenced. 

Unfortunately,  past  experience  has  shown  that  many  initial  deliverable  items  which 
are  being  produced  under  contract  to  the  government  are  less  than  adequate  training  tools 
and  do  not  meet  the  basic  requirements  of  the  DIDs  or  the  ISD  model.  There  exists  many 
possible  reasons  for  this  inadequacy  ranging  from  poorly  written  specifications,  and  thus 
different  interpretations,  to  a  lack  of  expertise  in  educational  foundations  and  technical 
writing.  From  a  review  of  various  training  packages  delivered  by  several  contractors,  one 
can  conclude  that  although  contractor  personnel  who  develop  training  material  possess  a 
great  deal  of  technical  expertise  and  subject  matter  knowledge,  many  do  not  apply  the  fun¬ 
damental  skills  of  education  theory  and  technical  writing.  This  results  in  poorly  written 
training  deliverables  and  ineffective  communication  concerning  educational  requirements. 

This  paper  will  highlight  and  investigate  the  problems  that  the  authors  have  experi¬ 
enced  in  the  area  of  evaluation  and  acceptance  of  technical  weapons  systems  training  mate¬ 
rial.  It  will  also  offer  suggestions  as  to  what  government  contractors,  as  well  as  the 
government,  may  do  in  order  to  produce  and  deliver  a  better  quality  product  in  a  much  more 
cost-effective  and  expeditious  manner. 


INTRODUCTION 

As  we  sit  down  to  write  this  article,  we 
have  just  returned  from  what  many  may  consider 
a  typical  government/contractor  progress  review. 
Perusing  our  notes,  it  becomes  apparent  that 
a  multitude  of  breakdowns  in  communication 
occurred  between  the  government  and  the  con¬ 
tractor  as  well  as  internally  within  both  the 
government  and  the  contractor's  plant.  These 
breakdowns,  unfortunately,  resulted  in  numer¬ 
ous  arguments,  increased  interpersonal  friction, 
and  the  creation  of  ever  present  government 
change  order <  causing  an  escalation  in  the 
price  of  the  contract. 

A  cursory  examination  of  a  successful 
contractor's  resources  reveals  that  the 
foundations  required  for  effective  curricula 
development  are  generally  present.  Therefore, 


the  question  must  be  asked:  Are  these  train¬ 
ing  foundations  properly  utilized  in  order  to 
meet  the  high  demands  of  the  Department  of 
Defense?  This  paper  will  examine  this  issue, 
offer  insights  into  current  vendor  produced 
training  materials  and  provide  suggestions 
for  improvements. 

In  evaluating  the  adequacy  or  inadequacy 
of  contractor-produced  training  materials,  it 
is  necessary  to  briefly  examine  the  procure¬ 
ment  process  and  highlight  the  potential 
anomalies  which  exist.  The  cycle  commences 
with  the  solicitations.  Simply  stated,  it  is 
here  that  the  government,  via  an  Invitation 
for  Bid  (IFB) ,  Request  for  Proposals  (RFP) , 
Request  for  Quotations  (RFQ) ,  etc.,  describes 
the  service  and/or  product  it  wishes  to  ac¬ 
quire.  Herein  lies  the  first  problem.  Surely 
you  have  experienced  the  difficulty  and 


511 


frustration  of  not  being  able  to  explain 
exactly  what  it  is  that  you  want.  This  level 
of  difficulty  can  vary  from  relatively  simple 
instructions  such  as  "turn  on  the  light  in 
the  living  room"  to  much  more  complex  com¬ 
munications  as  "describe  the  operation  of 
your  318  V8  turbocharger."  Imagine  the  numer¬ 
ous  communication  hurdles  which  must  be  over¬ 
come  by  government  personnel  in  describing  a 
training  program  for  a  device  or  system  which 
has  yet  to  be  conceived.  How  can  training 
element  managers  provide  explicit  details 
for  training  programs  related  to  a  novel 
system  when  the  capabilities  of  that  system 
cannot  be  sufficiently  identified  or  described 
by  representatives  of  the  user  in  the  field? 
This  is  one  basic  problem  faced  by  training 
element  managers  on  a  daily  basis. 

Take  a  step  back  and  look  at  the  existing 
state-of-the-art  in  current  weapons  systems 
development.  Compare  today's  defense  systems 
with  those  of  a  few  years  ago.  It  becomes 
readily  apparent  that  technology  has  advanced 
at  an  escalating  rate.  On  the  other  hand, 
pick  up  a  daily  newspaper  and  read  one  of  the 
articles  concerning  the  inability  of  many  of 
today's  high  school  graduates  to  read  and 
comprehend.  The  combination  of  more  complex 
and  sophisticated  technology  with  an  apparent 
decreased  reading  grade  level  has  created  a  • 
great  amount  of  concern  among  personnel  in 
the  operating  forces.  It  is  apparent  that  no 
matter  how  "wonderful"  and  sophisticated  a 
system  is ,  it  is  of  absolutely  no  value  unless 
it  is  properly  operated  and  maintained.  There 
then  exists  a  dichotomy  which  generates  a 
"hypothetical  gap  between  technological 
development  of  hardware  procured  by  DOD  and 
the  educational  accomplishments  of  the  target 
population  assigned  to  operate  and  maintain 
this  equipment."  (2)  Training  personnel 
within  the  government  have  recognized  this, 
and  in  the  mid  70' s  began  to  institute  a 
number  of  rather  dramatic  changes  to  both 
in-house  and  contractor-developed  training 
programs.  Now  all  programs  must  be  developed 
in  accordance  with  instructional  Systems 
Development  (ISD)  procedures.  ISD  is  basic¬ 
ally  a  total  systems  approach  to  training. 

That  is,  training  as  a  whole  is  analyzed  and 
initial  determinations  are  made  as  to  what 
constitutes  a  task.  These  tasks  are  then 
further  analyzed,  and  the  ones  selected  for 
training  arc  associated  with  an  objective  as 
well  as  an  evaluation  criterion  for  that 
objective.  When  this  has  been  determined, 
the  training  design  and  media  selection  are 
undertaken. 

The  ISD  model  is  basically  a  very  sound 
model  which  utilizes  many  of  the  highly 
theoretical  education  concepts  presented  to 
us  in  college  education  courses.  These 
theories  were  highly  idealogical  and  very 
rarely  worked  as  they  were  supposed  to  in 
tho  public  schools.  Nonetheless,  initial 
military  training  material  developed  under 
ISD  is  now  being  fielded  throughout  the 
military  spectrum  and  the  results  are  sur¬ 
prisingly  pleasant,  studies  indicate  that 
a  greater  amount  of  learning  transfer  is 
taking  place  via  ISD-produced  material  than 
by  older,  more  traditional  methods  of 


inatruction.  (1)  This  initial  evidence  haa 
caused  many  high  officials  in  tha  Dapartment 
of  Defense  to  take  a  second  and  third  look  at 
ISD.  Aa  a  result,  it  may  be  safe  to  predict 
that  ISD-devaloped  materials  will  be  getting 
greater  attention  and  will  continue  to  be 
required  for  training  material  development. 
Therefore,  it  would  be  of  great  benefit  to 
both  government  and  contractors  to  have  a 
deeper  understanding  of  the  current  state  of 
affairs  concerning  ISD. 

Unfortunetely ,  in  the  view  of  the 
authors,  industry  has  not  yet  achieved  a 
working  understanding  of  ths  basic  theory 
and  operation  of  the  ISD  model.  This  has 
caused  deliverables  to  be  in  non-conformance 
with  ISD  principles,  and  therefore,  unaccept¬ 
able  to  government  reviewers.  Private  con¬ 
tractors  seemingly  still  recruit  personnel 
familiar  with  the  technical  aspects  concern¬ 
ing  operation  and  maintenance  of  hardware 
but  neglect  to  hire  ISD  specialists.  Without 
innovative  training  technology,  the  result 
is  very  often  an  inferior  or  unacceptable 
product.  Let  us  examine  the  effect  that  this 
lack  of  ISD  understanding  has  on  training 
system  design. 


TASK  ANALYSIS 

The  first  step  in  the  ISD  process  is  to 
establish  what  constitutes  or  will  constitute 
adequate  on-the-job  performance.  This  is 
referred  to  as  a  Task  Analysis.  The  Task 
Analysis  should  serve  to  generate  the  criteria 
and  data  that  provide  a  basis  for  initial 
foundations  as  well  as  assist  in  the  selec¬ 
tion  of  alternative  concepts  and  designs 
without  constraining  creativity.  More  spe¬ 
cifically,  Task  Analysis  answers  the  ques¬ 
tion  of  what  tasks,  performed  in  what  manner, 
under  what  conditions,  in  response  to  what 
questions,  and  to  what  standards  of  perform¬ 
ance  make  up  the  job.  Regardless  of  how 
well  the  next  steps  are  carried  out,  if  the 
job  analysis  data  is  not  valid  and  reliable, 
the  resulting  instructional  program  will 
fail  to  produce  personnel  competent  to  per¬ 
form  their  duties  at  a  basic  level. 

Currently,  the  government  is  receiving 
far  from  adequate  task  analyses.  It  seems 
that  all  too  often  contractor  training  per¬ 
sonnel  do  not  understand  what  is  required  in 
order  to  construct  a  solid  task  analysis  or 
simply  do  not  have  access  to  the  necessary 
information  needed  for  decision  making.  One 
available  resource  which  contractor  training 
personnel  may  use  to  support  this  phase  of 
development  is  the  Logistic  Support  Analysis 
Report  (LSAR).  (5) 

The  LSAR,  which  is  suprlied  by  contractor 
project  engineers  and  logistic  managers,  pro¬ 
vides  a  breakdown  of  operation  and  maintenance 
support  activities  currently  being  experienced 
or  expected  to  be  experienced  by  fielded 
systems.  It  is  the  job  of  contractor  training 
personnel  to  analyze  this  data  and  make  recom¬ 
mendations  as  to  task  determination  and  the 
requirements  for  training.  Some  tasks  are 
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seldom  required  on  the  job  and,  only  minimum 
job  degradation  would  result  If  the  task 
were  not  performed.  On  the  other  hand,  acne 
tasks  are  highly  critical  to  successful  job 
performance,  or  the  complex  nature  of  the 
task  makes  training  essential.  Economic  and 
time  considerations  require  the  trainer  to 
make  decisions  as  to  which  tasks  will  be 
selected  for  training  and  the  extent  of 
training  which  will  be  provided.  Many  con¬ 
tractors  either  fail  to  realize  or  neglect 
to  consider  the  inqiortance  of  this  task  se¬ 
lection  process.  The  result  of  this  in¬ 
attention  to  proper  task  selection  destroys 
the  foundation  of  the  training  design  and 
results  in  the  production  of  an  unacceptable 
training  curriculum.  Figure  1  provides  an 
example  of  what  an  unacceptable  submission 
of  a  task  and  skill  analysis  would  be.  It 
is  apparent  from  thiB  example  that  the  cur¬ 
riculum  developers  either:  a) ,  did  not  have 
a  working  knowledge  of  education  concepts 
or  the  ISO  model;  or  b),  were  not  provided 
with  sufficient  information  via  preliminary 
research,  i.e.,  LSAR;  or  c) ,  did  not  take  an 
adequate  amount  of  time  to  prepare  the  doc¬ 
ument,  or  d) ,  simply  had  a  very  poor  writing 
ability. 


as  was  mentioned  earlier,  the  task  and 
skills  analysis  is  the  bane  of  the  develop¬ 
ment  effort.  It  is  the  foundation  upon  which 
the  curriculum  will  be  built.  If  this  section 
lacks  continuity  or  is  vague,  as  shown  in 
Figure  1,  the  whble  program  suffors.  Many 
developers  fail  to  realize  or  understand 
this,  as  evidenced  by  actions  such  as  deliv¬ 
ering  an  instructor's  manual  or  a  student's 
guide,  months  before  the  task  and  skill 
analyses.  When  asked  about  it,  they  reply, 

"It 'b  not  complete  as  yet,  but  will  be  com¬ 
pleted  soon."  It  equates  to  a  student  writing 
a  required  outline  after  the  completion  of  a 
term  paper.  Contractors  do  not  understand 
that  an  outline.  Task  and  Skill  Analysis,  is 
a  tool  which  helps  turn  out  a  professional 
product.  Figure  2  is  representative  of  an 
acceptable  analysis  which  supports  this 
initial  development  effort.  It  is  apparent 
that  the  developer  of  the  document  demon¬ 
strated  in  Figure  2  manifests  the  consid¬ 
erations  of  the  ISD  model  in  this  phase  of 
the  curriculum  development  process. 
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JOG  TASK  ANALYSIS  SUMMARY 


¥ 


Nomenclature 


FIRE  FIGHTING/OANAGE  CONTROL 


Date 


■■tt.MQYffeir  Wl. 


2. 


Task 

Ident. 


LI.  1 


TASK  STEPS  IDENTIFICATION  AND  ANALYSIS 


3. 

Step 

5. 

6. 

Task  Description 

Number 

Step  Description 

Equipment 

Operate  fire  fighting  equipment. 

LI.  1. 1 

Operate  the  P-250  puep. 

LI.  1.1.1 

Prepare  the  P-230  pump. 

LI. 1.1. 1.1 

Assemble  all  necessary  equip- 

Foot  valve 

•ent  In  the  operating  area. 

and  strainer 

/ 

LI.  1.1. 1.2 

Connect  the  foot  valve  end 

Suction  hose 

strainer  to  the  suction  hose. 

Exhaust  hose 

11.1.1.1.2.1 

Place  the  female  threads  of  th« 

Spanner 

wrench 

foot  valve  and  strainer  against 
the  male  threads  of  the  suction 

hose. 

Tri-gate 

LI.  1.1. 1.2. 2 

Give  the  foot  valve  and  strain¬ 
er  a  quarter  turn  counterclock¬ 
wise  to  align  the  threads. 

2-1/2  inch 
fire  hose 

LI. 1.1.1. 2. 3 

Turn  the  foot  valve  and  strain¬ 
er  clockwise  until  it  is  tight. 

Two  1-1/2 

Inch  hoses 

LI.  1.1. 1.3 

Connect  the  suction  hose  to 
tho  P-250  pump. 

Two  all  pur¬ 
pose  nozzles 

* 

Fuel  tank 

Screwdriver 

FIGURE  2 

This  ta  an  example  of  an  acceptable  portion  of  a  task  and  skill 
analysis.  Note  that  the  enabling  objectives  are  modified  by 
specific  job  performance  measures.  The  numbering  system  identifies 
a  topical  outline  format. 


As  government  representatives  tasked  with 
the  responsibility  of  accepting  and/or  reject¬ 
ing  the  deliverables  provided  by  private  con¬ 
tractors,  curriculum  reviewers  devote  a  great 
deal  of  time  to  the  evaluation  of  the  task 
and  skills  analysis.  Specifically,  they 
examine  the  logical  progression  of  the  tasks 
selected  to  determine  whether  the  tasks  are 
actually  tasks  and  if  so:  a)  do  they  satisfy 
a  need  for  training:  b)  can  training  be  pro¬ 
vided  in  a  practical  and  cost-effective  manner; 
c)  do  the  tasks  complement  each  other  for  use 
on  the  job;  and  d)  how  will  performance  mea¬ 
sures  be  constructed? 

The  area  of  performance  measures  provides 
an  interesting  observation.  A  surprising  num¬ 
ber  of  contractor  training  personnel  seem  to 
be  encountering  difficulties  in  the  con¬ 
struction  of  job  performance  measures.  Review 
meetings  between  government  and  contractor 


personnel  repeatedly  revealed  this  fact. 
Questions  pertaining  to  what  should  be  mea¬ 
sured,  i.e.,  product,  process  or  both,  via 
what  method,  and  with  what  consequences  of 
failure  often  were  the  cause  of  many  vague 
responses.  Inquiries  about  predictive  valid¬ 
ity  or  fidelity  seldom  were  answered  in  con¬ 
crete  terms.  Since  these  topics  were  ques¬ 
tioned  early  in  the  design  process,  it  often 
led  to  strains  between  contractor  and  govern¬ 
ment  personnel.  This,  unfortunately,  caused 
tension  to  develop  and  decreased  the  level 
of  interpersonal  communication  which  is 
greatly  needed  throughout  the  design-develop¬ 
ment  of  the  curriculum. 
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DESIGN 

The  second  phase  of  the  ISD  model  Is  that 
of  design.  Unfortunately,  the  reaction  by 
many  government  Training  Element  Managers,  as 
to  whether  or  not  the  curriculum  is  favorably 
received,  is  usually  based  solely  upon  how  well 
the  materials  are  initially  presented.  It  can 
be  compared  to  the  new  home  buyer  who  bases 
his  opinion  of  a  house  solely  on  curb  appeal  - 
new  paint,  nice  lawn,  etc.  Ideally,  government 
reviewers  must  go  beyond  that  and  examine  the 
foundation  as  welli  thus,  a  "pretty"  instructor 
guide  with  little  substantiation  for  its  con¬ 
tents  will  not  necessarily  meet  the  rigorous 
government  acceptance  standards. 

Learning  objectives,  which  are  the  ’'heart” 
of  the  design  phase,  are  reviewed  closely. 
Adequate  job  performance  measures  which  are 
developed  in  the  task  analysis  provide  the 
basis  for  developing  these  learning  objectives. 


Terminal  learning  objectives  are  actually 
direct  translations  of  jab  performance 
measures  (JPMs)  into  learning  objectives  for 
the  training  world.  Therefore,  if  the  de¬ 
signer  takes  time  to  prepare  a  well  construc¬ 
ted  task  analysis  which  contains  rather  de¬ 
tailed  JPMs,  the  design  of  th  :  associated 
materials  should  be  relativel  /  simple. 

Objectives  contain  specific  descriptions 
of  an  action  the  learner  is  to  exhibit  after 
training,  the  conditions  under  which  the 
action  will  take  place,  and  the  standards  or 
criteria  which  must  be  reached  for  satis¬ 
factory  performance.  These  objectives  are 
generally  behavioral  in  nature  and  are  con¬ 
cerned  with  various  types  of  learning  or 
performance.  The  philosophy  under  which  they 
and  all  other  curriculum  efforts  operate  are 
based  upon  numerous  education  theories  which 
baccate  very  abstract  in  nature.  It  is  nec¬ 
essary  to  hire  personnel  familiar  with  edu¬ 
cation  concepts  and  who  can  appreciate  the 
importance  and  usefulness  of  the  prerequisite 
research  and  development  work  that  must  be 
accomplished  prior  to  the  "painting"  of  the 
final  product;  that  is,  task  analysis. 


FIGURE  3b 


Unfortunately,  it  is  rare  that  the  gov¬ 
ernment  receives  deliverables  which  contain 
well-written  objectives.  This  is  illustrated 
in  Figure  3.  The  reasons  for  this  vary  with 
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Adjust  Voltags  Output*  In  Radar  Rscsivar-Tranamittar 
Power  Supply  of  RT-818/TPN-18 


CONDITIONS 

You  will  bs  requirsd  to  perform  this  task  in  •  fixed  or  tactical  facil¬ 
ity,  indoors  or  outdoors,  under  all  weather  conditions.  Caution  must 
be  observed  during  inclement  weather,  ,as  a  greater  chance  of  shock 
hazard  will  exist.  Supervision  is  not  required.  The  following  items 
will  be  needed  to  perform  this  task: 

1.  Equipment. 

Radar  Set,  AN/TPN-18. 

2.  Tools  and  Test  Equipment. 

a.  Alignment  tool. 

b.  Multimeter,  AN/USM-223  or  equivalent. 

3.  References  and  Forms. 

TM  11-5840-281-12. 

Rader  receiver-transmitter  power  supply  has  been  determined  to  be 
out  of  adjustment  by  previous  troubleshooting  tasks  that  have  been 
performed . 


STANDARDS 

Job  standard  has  bsen  mat  whan  tha  voltage  outputs  or  ths  radar 
receiver-transmitter  power  supply  have  been  adjusted  to  -35  V  dc  for 
the  -35  volt  power  supply,  +35  V  dc  for  ths  +35  volt  power  supply, 
♦100  V  dc  for  the  +100  power  supply,  and  +200  V  dc  for  tha  +200  volt 
powar  supply  in  accordanct  with  performance  measures  3  through  14. 
Performance  measures  1  through  16  should  be  completed  within  10 
minutes. 
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TERMINAL  OBJECTIVE: 


*9* 


The  student  will  be 
able  toCnanipulatt 
the  I/O  via  the 
maintenance  panel 


a.  ftp**-- 


FIGURE  3a 


Figure  3a  ie  example 
of  an  unacceptable 
objective  actually  sub¬ 
mitted  for  review. 

Figure  Zb  ie  an  example 
of  a  correctly  written 
objective.  Note  that 
here  all  three  elements, 
action,  condition  and 
standard  are  included,  in 
thie  example  ae  required 
by  the  ISD  model. 


such  things  as  contract  requirements,  the 
philosophies  of  the  contractors  as  well  as  the 
government  Training  Element  Managers,  overly 
specific  or  extremely  vague  data  item  descrip¬ 
tions  (DIDs)  and  writer  inability  or  misunder¬ 
standing.  Let  us  examine  this  a  bit  further. 

One  of  the  major  dilemmas  which  has  a 
direct  effect  upon  the  design  of  instructional 
materials  is  legal  requirements  of  contractor 
scheduling.  The  contracting  officer  places 
a  great  deal  of  emphasis  upon  delivery  dates 
as  is  necessary  to  meet  the  terms  cf  a  con¬ 
tract.  Tills  is  often  dictated  by  such  things 
as  operational  equipment  delivery,  ready  for 
training  dates,  etc.,  which  are  established 
by  higher  headquarters,  thus  placing  any 
decision  out  of  the  realm  of  working  level 
personnel.  Unfortunately,  this  causes  the 
contractor  to  be  forced  into  an  obsession  to 
comply  -  delivery  on  time  becomes  the  striving 
goal.  As  a  result,  the  quality  of  the  program 
is  often  overshadowed  by  the  delivery  date, 
and  the  government  receives  numerous  blank 
pages  titled  and  noted  "to  bo  developed" 
because  of  insufficient  data.  This  is  extreme¬ 
ly  frustrating,  as  well  as  very  costly. 

Perhaps,  there  should  be  a  re-examination 
as  to  the  legitimacy  of  scheduling  requirements 
and  the  applicability  of  data  item  descriptions 
or  other  specific  working  standard?  before  any 
development  of  curricula  is  undertaken.  It  is 
apparent  that  something  needs  to  be  done  in 
order  to  focus  the  attention  toward  quality 
research  and  design  rather  trian  compliance 
with  delivery  dates  juet  to  satisfy  legal 
requirements  in  the  contracts. 

Corporate  philosophies  al.no  affect  the 
training  product.  Technology  growth  in  the 
last  ten  years  has  been  astonishing.  Atti¬ 
tudes  toward  training  design  have  arso 
changed  dramatically.  Although  the  basic 
foundations  of  education  are  still  the  same, 
the  training  methodology  has  takeR  on  new 
dimensions.  Inno-ative  government  education 
and  training  specialists  have  realized  this 
and  are  experimenting  with  novel  approaches 
to  training  Co r  their  programs.  Such 
vehicles  as  videotape  and  videodisc,  computer 
assisted  instruction  as  well  as  visual  and 
physical  simulation,  art  being  solicited  in 
new  weapons  systems  procurement.  The  reasons 
for  this  can  be  capsulized  into  the  fact  that 
training  via  these  avenues  is  more  efficient, 
interesting,  cost-effective  and  practical. 

This  is  not  to  say  that  the  traditional 
written  material  is  to  become  obsolete.  There 
will  always  be  a  requirement  for  written  doc¬ 
umentation  and  instruction.  Fortunately,  as 
many  innovative  contractors  are  beginning  to 
realize,  these  simple  written  materials  must 
be  augmented  by  innovative  training  devices. 

The  Department  of  Defense's  training  require¬ 
ments  have  become  so  technical,  complex  and 
expensive  that  now  the  most  efficient  method 
of  training  is  often  via  non-traditional 
methods  similar  to  those  mentioned  earlier. 

It  is  the  opinion  of  these  authors  that  the 
time  has  now  arrived  that  if  a  contractor 
does  not  soon  make  a  decision  to  hire  com¬ 
petent  personnel  who  are  well  versed  in  the 


non-traditional  avenues  of  trainingi  i.e., 
simulation,  video/computer  interface,  etc,, 
he  will  fall  by  the  wayside  in  his  quest  to 
obtain  defense  systems  contracts. 

In  essence,  then,  there  must  be  certain 
changes  made  in  the  training  acquisition 
process.  Initially,  the  government  must  pro¬ 
vide  more  specific  direction  to  the  contrac¬ 
tor  regarding  the  product  which  is  to  be 
developed.  In  addition,  it  is  imperative 
that  the  contractor  be  given  the  freedom  to 
determine,  a)  what  is  needed  for  training, 
b)  the  most  effective  methods  to  provide 
that  training,  and  c)  adequate  time  for  the 
preparation  of  the  instructional  material. 

In  summary,  this  paper  has  pro  ided  a 
cursory  examination  of  the  current  process  of 
training  system  acquisition.  There  exist 
many  problems,  ranging  from  minute  to  menu- 
mental,  caused  by  both  the  goverunent  and 
contractors.  It  would  take  a  dissertation  to 
expound  upon  all  of  the  anomalies  which  are 
present.  Obviously,  tins  is  beyond  the  scope 
of  this  paper.  Figure  4  is  a  capsuled  attempt 
to  highlight  areas  where  deficiencies  are  per¬ 
ceived  to  exist.  It  is  structured  so  as  to 
remain  objective  in  nature,  yet  be  highly 
critical  of  both  parties  in  DOD  training 
acquisitions.  The  ISD  model  serves  as  its 
base,  since  it  is  the  vehicle  by  which  we,  as 
government  Training  Element  Managers,  review 
contractor  produced  deliverables. 

On  the  other  hand,  a  private  contractor 
whose  business  is  training  development  muBt 
be  aware  of  the  current  state-of-the-art  in 
education.  He  must  realize  that  trends  in 
technology  are  vastly  outpacing  those  in 
education  and  that  attempts  must  be  made  to 
bridge  this  hypothetical  gap.  This  can  be 
done  by  recruiting  personnel  who  are  know¬ 
ledgeable  about  current  trends  and  are  willing 
to  step  out  and  take  the  necessary  initiative 
to  introduce  the  innovative  training  method¬ 
ology  dictated  by  current  technological  ad¬ 
vances.  (1) 

We  do  not  purport  that  an  individual 
well  versed  in  ISD  is  the  panacea  for  in¬ 
adequate  training  program  development.  Ob¬ 
viously,  there  is  no  one  perfect  method  which 
will  ensure  effective  training.  But,  as 
Montemerlo  and  Harris  suggest,  the  develop¬ 
ment  of  effective  training  packages  requires 
among  other  things  an  Interdisciplinary  team 
of  subject  matter  experts  and  skilled  in¬ 
structional  technologists.  (4)  The  recom¬ 
mendations  alluded  to  in  this  paper  are  just 
as  susceptible  to  inept  application  as  are 
current  methods.  The  authors  suggest  that 
if  training  development  is  given  additional 
attention,  and  conscientious  consideration, 
there  will  be  a  significant  impact  upon 
future  training  developments. 


516 


SUGGESTED 

GOVERNMENT 

ACTION 

Task  Selection/Deter-  Provide  adequate  job 
initiation  description/constraints. 


for  Insure  contractor  under¬ 
stands/develops  logical 
rationale  for  selection. 


Select  Tasks 
N  Training 

A 
L 
Y 
S 

I 

s 

Write  Job  Performance 
Measures 


D  Objectives 

E 

S 

I 

G 

Tests 

N 


Validation 


Provide  regulatory  in¬ 
struction  (DIDs)  which 
demonstrate  concrete 
guidelines  and  do  not 
stifle  contractor  creati¬ 
vity. 


Must  insure  that  salient 
points  of  objectives  are 
tested.  Insure  written 
and  performance  eval¬ 
uations  have  predictive 
validity. 


Should  inform  contrac¬ 
tor  of  desired  outcome 
of  product;  i.e.,  what 
they  expect  the  product 
to  accomplish. 


SUGGESTED 

CONTRACTOR 

ACTION 

Analyze  LSARs,  etc. 
and  make  determina¬ 
tions  for  ISD  incorpo¬ 
ration. 

Examine  complete  pic¬ 
ture  as  to  need,  practi¬ 
cality  and  cost. 


Provide  rationale  which 
may  determine  evalua¬ 
tion  of  tasks. 


Construct  objectives 
which  contain  all  ele¬ 
ments  (Action,  Condi¬ 
tion,  Standard)  and  are 
criterion  referenced  as 
well  as  performance 
based  whenever  possi¬ 
ble. 

Insure  all  facets  of  ob¬ 
jective  are  adequately 
measured. 


Examine  target  popula¬ 
tion  as  well  as  subject 
matter  and  make  deter¬ 
mination  as  to  most 
efficient  media  for  in¬ 
formation  presentation. 


Design  evaluation  pro¬ 
cedure  before  material 
is  fielded;  i.e.,  inform 
the  government  as  to 
the  methodology  which 
will  be  employed  in 
determining  if  a  product 
is  reliable  and  valid. 


CURRENT  PRODUCT 


Tasks  not  definitive. 


Entire  instructional 
system  not  considered 


Not  being  done  prior  to 
design  of  material  as 
evidenced  by  lack  of 
evaluation  criteria. 


Objectives  generally  not 
acceptable  due  to  lack 
of  one  or  more 
elements.  Often  numer¬ 
ous  behaviors  are  con¬ 
tained  in  one  objective 
and  performance  mea¬ 
sures  are  vague. 

Tests  are  constructed 
after  instruction  is  writ¬ 
ten;  result  is  a  simple 
reiteration  of  classroom 
presentation. 


Little,  if  any,  target 
population  analysis 
done.  Training  material 
currently  consists  pri¬ 
marily  of  traditional 
written  manuals  which 
impact  upon  learning 
transfer  due  to  stu¬ 
dent’s  comprehension  of 
material. 

No  real  guidelines  or 
procedures  which  may 
determine  if  a  product 
(curriculum)  accom¬ 
plishes  its  objectives. 


Media  Selection/Format  State  of  the  art  in  train¬ 
ing  must  be  considered 
and  leeway  for  contrac¬ 
tor  creativity  allowed. 
COTRs  cannot  be  reluc¬ 
tant  to  introduce  inno¬ 
vative  design. 


FIGURE  4 

Summary  of  deliverables  relative  to  the  ISD  model. 


SUGGESTIONS 


Contractor  insure  proper  inter¬ 
face  and  understanding  between 
LSA  and  ISD  personnel. 


Contractor  should  take  further 
steps  to  analyze  target  popula¬ 
tion  and  training  environment. 
Government  should  provide 
more  definitive  guidance  as  to 
the  product  they  desire;  ex¬ 
amples  should  be  provided. 
Government  should  also  pro¬ 
vide  adequate  preparation  time 
for  foundations  of  training. 
Quality  suffers  if  this  phase  is 
eliminated  and  COTR  should 
budget  accordingly. 

Contractor  should  make  all  ef¬ 
forts  to  recruit  personnel 
familiar  with  training  evalua¬ 
tion  and  task  selection  techni¬ 
ques. 


Contractor’s  viewpoint  needs 
alteration.  Behavioral  objective 
approach  is  often  viewed  as  a 
necessary  evil  and  not 
understood.  Consequently,  end 
product  generally  unsatisfac¬ 
tory. 


Contractor  should  employ 
specialists  in  the  area  of  test 
construction.  Curriculum 
should  not  just  provide  instruc¬ 
tion  necessary  to  “pass  the 
test.” 

Contractor’s  should  strive  to  in¬ 
troduce  more  innovative  and 
creative  training  methodologies. 
Visual  literacy  of  target  popula¬ 
tion  must  be  examined  and 
modern  techniques  employed. 


Experimental  test  design  pro¬ 
cedure  for  validation  be  con¬ 
structed  and  delivered  to 
government  prior  to  delivery  of 
material  to  the  field. 
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TRAINING  THE  FIELD  ARTILLERY  CANNON  SYSTEM 


Abstract 

Field  Artillery  units  are  preparing  for  caflbat  in  a  training  environment  that  is  often 
characterized  as  hostile.  One  of  the  factors  contributing  to  this  difficult  training  environment 
is  the  availability  of  training  resources  needed  to  achieve  and  maintain  a  high  state  of  combat 
readiness.  These  resources  include  personnel,  budget,  time,  fuel,  repair  parts  and  a  suitable 
location  to  conduct  realistic  training. 


The  field  artillery  is  not  without  training 
devices  and  simulators  at  present;  however,  in 
the  past  the  philosophy  has  been  that  the  best 
way  to  train  field  artillerymen  is  with  live 
anmunltion.  While  a  strong  case  can  be  made  to 
support  that  philosophy,  and  while  adequate 
resources  in  the  past  have  permitted  that  philo¬ 
sophy  to  reign,  the  recent  dramatic  cost  in¬ 
creases  for  these  resources  are  driving  the  Army 
to  do  things  differently.  These  changes  have 
been  recognized  and  are  being  dealt  with  by  the 
Field  Artillery  School. 

The  Army  and  the  Field  Artillery  are  heav¬ 
ily  committed  to  the  use  of  training  devices  as 
a  method  of  simulating  force-on-force  combat. 
As  technology  advances  and  the  understanding  of 
the  factors  required  for  effective  training 
increases,  devices  continue  to  be  one  of  the 
best  solutions  for  economically  and  realisti¬ 
cally  conducting  combat  training. 


WHY  WE  NEED  TRAINING  DEVICES  AND  SIMULATIONS 

Field  Artillery  units  are  preparing  for 
combat  in  a  training  enviroment  which  is 
characterized  as  hostile  and  ever-shrinking. 
One  of  the  several  factors  contributing  to  this 
difficult  environment  is  the  availability  of 
training  resources  needed  to  achieve  and  main¬ 
tain  a  high  state  of  combat  readiness.  Spiral¬ 
ing  costs  for  ammunition,  fuel,  and  repair 
parts,  combined  with  limited  training  areas  and 
available  training  time  are  resources  which  are 
becoming  scarce.  This  trend  will  continue  over 
the  coming  years. 

During  the  period  1977  to  1980,  the  cost  of 
105imn  and  155mm  ammunition  Increased  91Z,  while 
the  cost  of  an  8-inch  full-service  round  has 
Increased  by  23 4X  during  the  same  period.  As  a 
result,  ammunition  allocations  for  training  of 
active  as  well  as  reserve  component  field  artil¬ 
lery  units  have  been  cut  significantly,  and 
future  reductions  in  training  ammunition  allo¬ 
cations  should  be  expected. 

The  "energy  crunch"  and  the  associated  high 
fuel  costs  have  had  an  adverse  impact  on  avail¬ 
ability  of  fuel  to  support  training.  The  cost 
for  mogas  and  diesel  has  Increased  by  over  20035 
since  197  7.  Using  an  annual,  average  expendi¬ 
ture  of  40,000  gallons  of  diesel  and  15,000 
gallons  for  mogas  for  training,  a  155mm  FA 
battalion  annually  must  pay  $37,000  more  for 
diesel  and  $13,000  more  for  mogas  today  than  it 
did  in  1977. 

Repair  parts  have  also  increased  in  cost 
dramatically  over  the  past  4  years.  In  1977 
average  annual  repair  parts  costs  for  a  CONUS  FA 
battalion  were  $388,000.  Today  the  cost  is 
approximately  $600,000, 


The  adequacy  and  availability  of  training 
areas  are  other  factors  that  contribute  to  the 
hostility  of  the  training  environment.  Fre¬ 
quently,  those  training  areas  that  are  adequate 
are  heavily  ccmmltted  and,  in  many  cases  safety 
considerations  for  many  special  munitions  mean 
their  use  in  training  is  restricted  to  a 
very  few  firing  ranges.  Additionally,  environ¬ 
mental  considerations,  particularly  noise 
pollution,  are  becoming  Increasingly  Important. 
Civilian  communities  in  the  vicinity  of  artil¬ 
lery  Impact  areas  are  more  concerned  about  the 
noise  of  impacting  artillery  than  they  were  ten 
years  ago,  and  this  has  caused  Increasing  pres¬ 
sures  to  restrict  the  availability  of  training 
areas. 


TRAINING  THE  CANNON  SYSTEM 

There  are  several  ways  to  look  at  field 
artillery  cannon  unit  training  when  designing  a 
family  of  training  devices  and  simulations  to 
complement  live  fire  training.  Traditional 
approaches  involve  consideration  of  the  elements 
of  a  firing  battery,  i.e,,  the  fire  direction 
center,  the  forward  observer,  and  the  howitzers 
themselves  as  a  way  of  analyzing  or  expressing 
the  need  for  training  devices  and  simulation. 
It  is  also  possible  to  look  at  training  device 
and  simulation  needs  in  terms  of  individual 
training  and  collective  training  requirements. 
These  perspectives  have  several  fundamental 
flaws,  however,  which  may  have  contributed  to 
the  current  state  of  affairs.  There  is  a  more 
fundamental  perspective  that  appears  to  yield 
better  insights  into  how  field  artillery  units 
can  take  advantage  of  training  devices  and  simu¬ 
lation.  This  perspective  involves  consideration 
of  where  field  artillery  units  train,  what  type 
of  training  is  conducted  there,  what  training 
devices  or  simulations  are  currently  available 
to  support  this  training,  and  idiat  training 
devices  and  simulations  should  be  developed  to 
Improve  the  training  conducted  there  at  less 
cost  with  no  loss  in  effectiveness. 

Training  is  normally  conducted  by  FA  units 
in  three  training  areas:  garrison  areas,  local 
or  closein  training  areas,  and  major  training 
areas  (MTA) . 

1.  Garrison.  Field  Artillery  related  garrison 
training  is  characterized  as  focusing  on  devel¬ 
opment  of  individual  skills  and  section  level 
proficiency.  Normally  the  training  is  done  in  a 
classroom,  or  the  motor  pool,  and  seldom  do  the 
sections  integrate  their  training  efforts.  This 
mode  of  training  usually  follows  ARTEP  type 
training  tdiere  the  commander  identifies  his 
training  weaknesses,  diagnoses  the  cause  of  each 
weabtess,  and  prescribes  training  at  individual 
or  section  level  to  Improve  proficiency  prior  to 
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the  next  evaluation.  Garrison  type  training 
because  of  these  characteristics  Is  by  Its 
nature  less  expensive  than  training  In  the  other 
a-eas.  Therefore,  training  devices  and  simula¬ 
tions  here  should  make  a  contribution  toward 
improving  effectiveness  of  training. 

At  present,  there  are  several  training  de¬ 
vices  and  simulations  in  the  inventory  which  can 
be  used  to  train  in  garrison: 

105  Ammo  Handler's  round 

Time  Training  Fuze 

Artillery  Direct  Fire  Trainer 

Nuclear  Training  Rounds  M455  155am, 

M423  8-in 

Training  Set,  Fire  Observation 

Puff  Board 

A  variety  of  Battle  Simulations  (war  games) 

As  can  be  seen,  this  is  not  an  Impressive 
array  of  training  devices  and  simulations  con¬ 
sidering  the  current  state  of  the  art,  and  the 
list  highlights  the  earlier  statement  that  field 
artillery  device  and  simulation  development  has 
been  done  on  a  piecemeal  basis. 

In  the  light  of  the  type  training  conducted 
in  garrison  and  the  training  devices  and  simu¬ 
lations  available  to  support  that  training, 
several  gaps  can  be  identified  which  provide  the 
basis  for  training  devices  or  simulations  cur¬ 
rently  under  development  or  for  establlshnent  of 
a  training  device  requirements  document. 

Training  devices  and  simulations  currently 
under  development  to  support  garrison  training 
are: 

a.  Copperhead  Training.  Round:  This  de¬ 

vice  will  provide  cannoneers  with  realistic  crew 
drill  on  a  dummy,  full-caliber  155mm  cannon- 
launched  guided  projectile.  The  round  does  not 
leave  the  tube  and  must  be  extracted  through  the 
breech.  Crew  dr*'  '  will  include  inspection, 
arming  and  loadin  . . 

b.  Battery  Computer  System  (BCS)  Trainer: 
This  trainer  Is  being  designed  as  a  means  of 
conducting  operator  training  on  the  BCS  while 
simulating  the  interface  with  systems  having  an 
input  to  BCS  (TACFIRF.,  Digital  Message  Device. 

c.  Training  Set,  Fire  Observation  (TSFO) : 
This  training  device  is  designed  to  provide 
realistic  classroom  training  in  adjust  fire  pro¬ 
cedures,  both  for  institutional  and  unit  use. 
It  utilizes  image  projectors  and  a  minicomputer 
to  display  target  a»d  bur;  'ibols  on  a  slide- 
projected  te-  •"?-  ne.  T  i  be  fielc  I  two 
per  division  -iii.il  lery  i  us  selected  TRADOC 
schools  and  reserve  component  locations, 

d.  Ground  Laser  Locator  Designator  Eval¬ 
uator  (GLLD-E)  :  This  device  is  designed  to 

allow  FIST  personnel  to  practice  GLLD  tracking 
without  the  safety  hazards  o  actual  GLLD. 

It  incorporates  a  Maverick  "  ^mera  and  pro¬ 
vides  feedback  to  the  ope\  :-jr  via  a  scoring 
device.  In  addition  to  these  devices  currently 
under  development,  there  is  a  need  for  addi¬ 
tional  devices  to  fill  other  existing  gaps  in 


training  the  cannon  system  in  the  garrison  en¬ 
vironment:  for  example,  devices  which  will 
enable  the  total  field  artillery  system  to  train 
as  a  team  without  the  expenditure  of  service 
ammunition.  This  means  of  training  the  total 
system  is  seen  as  a  way  of  closing  the  loop 
between  the  observer  and  firing  battery  in  dry 
fire  training  and  may  be  viewed  as  a  long  term 
replacement  to  the  M31  trainer.  It  would  also 
enable  units  to  train  with  minimum  expenditures 
of  fuel  because  they  would  not  be  forced  to 
travel  to  maneuver  and  impact  areas.  An  over¬ 
view  of  how  this  integration  of  devices  might 
take  place  is  outlined  below: 

a.  The  Fire  Support  Team  (FIST)  trains 
with  the  Training  Set,  Fire  Observation  in  gar¬ 
rison.  A  direct  communication  link  is  estab¬ 
lished  with  the  TACFIRE  or  Battery  Computer 
System  element  of  the  battalion  and  battery  Fire 
Direction  Center  in  the  motor  pool. 

b.  The  Fire  Direction  Center  (battery  or 
battalion)  computes  firing  data  and  sends  it  to 
the  guns,  collocated  with  the  FDC  in  the  motor 
pool  or  local  training  area. 

c.  The  gun  sections  load  a  type  of  dummy 
projectile  while  error  measuring  equipment  on 
the  howitzers  feeds  data  to  a  central  computer 
which  determines  "did  hit"  coordinates. 

d.  A  burst/flash  symbol  is  displayed  on 
the  TSFO  screen  at  the  "did  hit"  coordinates. 
(Note:  The  central  computer  could  be  the  TSFO 
computer,  BCS,  TACFIRE,  or  a  separate  "training- 
only"  computer.) 

e.  A  data  link  from  the  central  computer 
to  target  acquisition  systems  provides  "did  hit" 
coordinates,  trajectory  information,  and  burst 
symbols  on  data  display  units. 

2.  Local  Training  Area  (LTA) .  Training  in  the 
LTA  primarily  concentrates  on  development  of 
collective  level  ARTEP  tasks  at  the  battery  and 
battalion  level.  For  many  CONUS  units,  the  LTA 
and  MTA  may  be  one  and  the  same  training  area. 
For  units  stationed  overseas,  there  is  a  marked 
difference  between  these  two  areas.  The  LTA  is 
generally  characterized  as  limited  in  space  but 
large  enough  to  allow  units  to  conduct  some 
training  in  reconnaissance,  selection,  and  occu¬ 
pation  of  position.  Any  live  firing  for  artil¬ 
lery  units,  if  permitted,  is  restricted  to 
subcaliber  devices.  Training  in  this  area 
builds  upon  the  individual  and  section  level 
proficiency  obtained  during  the  garrison  train¬ 
ing,  and  allows  the  command,  control,  and  coor¬ 
dination  function  to  be  exercised  at  the  battery 
and  battalion  level.  Some  training  devices  and 
simulations  can  also  be  used  in  the  LTA  as  units 
train  to  a  high  level  of  proficiency  prior  to 
conducting  live  fire  training  at  the  MTA. 

Tw>  field  artillery  devices  currently  used 
in  this  environment  include  the  M31,  14.5mm  sub¬ 
caliber  trainer  and  the  miniature  moving  target 
(a  Soviet  model  tank  used  in  conjunction  with 
M31  ranges).  At  present,  there  are  no  devices 
under  development  specifically  designed  to  en¬ 
hance  training  in  the  local  training  area.  Some 
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other  devices  mentioned  previously  for  use  in 
garrison  areas  can  also  be  used  in  the  LTA 
(lOSnn  ammo  handler's  round,  artillery  direct 
fire  trainer,  and  nuclear  training  rounds). 

The  Integration  of  devices  to  enable  the 
total  field  artillery  system  to  train  as  a  team 
without  the  expenditure  of  service  anmunition 
(discussed  under  "garrison"  training)  would  also 
be  an  excellent  means  of  training  the  battery/ 
battalion  collective  tasks  in  the  local  training 
area  (LTA).  It  would  allow  the  units  to  inte¬ 
grate  the  firing,  maneuver,  and  RSOP  tasks 
without  expending  service  ammunition.  In  the 
LTA,  one  additional  device  is  needed: 

A  low  cost,  full  caliber  training  projec¬ 
tile  which  can  be  rammed/ loaded  into  105mm, 
155mm,  and  8-inch  howitzers,  provide  bang  and 
recoil,  and  either  consumes  itself  or  literally 
plops  out  of  the  end  of  the  tube  so  that  dry 
fire  training  for  howitzer  crews  can  be  more 
realistically  done  while  not  creating  safety 
constraints  which  would  require  Impact  areas. 
This  round  would  also  be  used  during  engagement 
simulation  exercises.  A  round  which  can  be 
ranmed  but  must  then  be  extracted  through  the 
breech  does  not  provide  for  positive  training 
responses. 

3.  Major  Training  Area  (MTA) .  Training  in  the 
MTA  is  characterized  by  live  firing  with  service 
ammunition  within  the  framework  of  battery  and 
battalion  level  ARTEP  tasks.  In  some  MTA  loca¬ 
tions,  field  artillery  units  are  able  to  par¬ 
ticipate  in  live  fire  combined  arms  exercises. 
This  however,  is  the  exception  rather  than  the 
rule,  due  to  the  many  safety  restrictions  in 
force  at  the  various  training  areas. 

The  primary  emphasis  in  the  rlTA  is  use  of 
actual  combat  equipment  and  munitions  in  simu¬ 
lated  combat  scenarios.  Pew  training  devices 
are  used  during  this  "verification  of  combat 
readiness"  training.  One  exception  in  the 
future,  however,  will  be  the  use  of  full  cali¬ 
ber,  training-unique  ammunition.  The  field 
artillery  is  currently  developing  a  low-cost 
indirect  fire  training  round  (LITR)  to  be  used 
in  lieu  of  high  explosive  service  ammunition  in 
training  at  the  MTA's.  These  non-exploding 
indirect  fire  training  rounds  will  significantly 
reduce  the  cost  of  live  fire  training  for  155mm 
and  8-inch  battalions  and  institutional  train¬ 
ing.  The  rounds  ace  ballistically  matched  to 
their  parent  HE  round;  the  FDC  uses  the  same 
firing  tables  as  with  the  HE  round;  and  a  smoke/ 
flash  signature  is  provided  so  the  observer  can 
see  and  adjust  them. 

There  is  another  major  type  of  training  to 
be  conducted  in  the  near  future  at  the  MTA's  and 
that  is  engagement  simulation  exercises  using 
the  Multiple  Integrated  Laser  Engagement  System 
(MILES).  This  type  of  training  will  allow  free- 
play,  force-on-force  scenarios  to  be  conducted 
with  a  degree  of  battlefield  realism  never 
before  obtainable.  The  use  and  effects  of  live 
ammunition  are  effectively  simulated  with  the 
MILES  laser  transmitters  and  detectors.  The 


current  state  of  development  of  MILES  equipment 
is  primarily  limited  to  direct  fire  weapons 
systems,  A  large  training  gap  exists  which 
indicates  a  need  for  devices  and  methods  to 
realistically  play  indirect  fire  systems  in 
MILES  exercises,  both  st  the  National  Training 
Center  and  other  installations  having  MILES 
equipment. 

Five  distinct  elements  of  the  indirect  fire 
problem  have  been  identified  below  and  priori¬ 
tized  according  to  their  Importance  in  develop¬ 
ment  : 


a.  An  audio/ visual  cue  to  be  activated  at 
the  simulated  grid  of  Impact.  This  cue  (smoke, 
bang,  flash)  does  not  need  to  simulate  every 
round  fired, .  but  a  representative  number  is  re¬ 
quired  to  give  the  Artillery  Observer  and  other 
players  an  appreciation  of  the  suppression/ 
killing  power  of  their  supporting  artillery/ 
mortars  and  to  insure  that  the  attacked  unit  is 
clearly  aware  of  why  it  suffered  the  casualties/ 
damages  it  did  and  what  preventive  measures 
could  or  should  have  been  taken. 

b.  An  automatic  casualty  assessment  sys¬ 
tem  which  is  not  dependent  upon  the  subjective 
decisions  of  a  fire  marker  controller  on  the 
battlefield  with  a  controller's  laser  gun.  This 
system  must  automatically  assess  casualties  and 
damage  according  to  the  distance  from  the  Impact 
grid  of  the  simulated  rounds,  the  type  of  muni¬ 
tions  employed,  and  the  protection  available  to 
the  players  at  the  moment  of  impact. 

c.  A  means  of  getting  the  firing  battery 
and  FDC  realistically  involved  in  the  exercise. 
The  overview  of  an  integration  of  devices  de¬ 
signed  to  train  the  total  FA  system,  described 
earlier,  may  also  be  applicable  here.  The  "did 
hit"  coordinates  determined  by  the  central  com¬ 
puter,  based  upon  the  FDC' 8  computations  and 
subsequent  data  applied  to  the  howitzers,  could 
be  used  as  the  basis  for  the  casualties  assessed 
on  the  battlefield.  In  this  manner,  the  firing 
battery  and  FDC  personnel  have  direct  input  to 
the  success/  failure  of  the  field  artillery  in 
engagement  simulation  exercises.  In  conjunction 
with  the  error  measuring  equipment  on  the  howit¬ 
zers,  the  dummy  projectile,  described  earlier, 
could  be  used  to  increase  the  realism  of  engage¬ 
ment  simulation  by  the  crews. 

d.  A  GLLD  simulator  which  will  allow 
Copperhead  and  Hellfire  to  be  played  in  MILES 
exercises.  A  trainer  which  looks  like  a  GLLD  or 
an  adaption  to  the  actual  GLLD  itself  must  be 
developed  which  will  Interface  with  existing 
Miles  equipment  so  that  FIST  personnel  can  real¬ 
istically  simulate  the  coordination  and  laser 
designating  skills  involved  in  the  employment  of 
laser-guided  munitions. 

e.  A  means  of  realistically  involving  the 
various  target  acquisition  elements  into  the 
exercise.  Again,  the  earlier  discussion  on  the 
overview  of  an  integration  of  devices  designed 
to  train  the  total  FA  systsn  may  have  appli¬ 
cation  in  solving  this  need. 
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CONCLUSIONS 


Ths  following  point*  should  be  highlighted 
concerning  the  development  of  devices  and  simu¬ 
lations  to  train  ths  field  artillery  in  the 
1980's: 

1.  This  paper  addresses  training  device  needs 
which  have  been  Identified  and  should  not  be 
confused  with  rsquiraaent*.  Every  need  must 
undergo  a  thorough  front  end  analysis  (to  In¬ 
clude  task  analysis,  madia  selection  analysis, 
and  cost  ahd  training  ‘effectiveness  analysis) 
before  a  legitimate  .requirement  can  be  substan¬ 
tiated  and  requirements  documents  written. 

2.  This  plan  is  not  ;  be  considered  sn  all 
inclusive  listing  of  training  devices/slaula- 
tiona.  As  front  end  analyses  for  new  ccmbat 
systems  are  completed,  additional-  device  re¬ 
quirements  will  be  identified.  In  addition, 
TRADOC’s  Comprehensive  Plan  for  Training  Device 
Developments  is  a  listing  of  every  device 
currently  in  the  inventory,  under  development, 
and  proposed  for  future  development.  It  is 
updated  on  an  annual  basis. 

3.  Combat  systems  which  have  computers  as  part 
of  their  equipment  are  prime  candidates  for  the 
development  of  software  embedded  training.  This 
kind  of  training  reduces  the  requirement  for 
separate,  costly  training  devices/simulators  and 
also  permits  soldiers  to  train  and  be  tested  on 
their  own  combat  equipment. 

4.  A  prime  consideration  in  the  development  of 
training  devices  is  not  to  train  personnel  at 
the  institution  with  devices  that  will  not  be 
available  at  their  unit.  However,  this  should 
not  become  a  stumbling  block  to  the  development 
of  cost  efficient  training  at  the  institution. 
There  are  times  when  it  is  cheaper  or  more 
effective  to  develop  an  institutional -only 
trainer  due  to  space  constraints  ir  the  actual 
combat  equipment,  large  numbers  of  students  to 
be  trained  in  short  periods  of  time,  or  insuf¬ 
ficient  quantities  of  combat  equipment  available 
to  the  institution. 

5.  The  development  of  devices  to  integrate  the 
training  of  the  total  cannon  system  and  target 
acquisition  systems  without  the  expenditure  of 
service  ammunition  is  of  prime  importance,  both 
for  unit  collective  training  programs  and  for 
combined  arras  training  exercises  using  engage¬ 
ment  simulations. 

BIOGRAPHICAL  SKETCH 

Captain  Arthur  A,  Shrader 
Chief,  Training  Devices  and  Simulations  Branch 
Directorate  of  Training  Developments 
US  Army  Field  Artillery  School 

Fort  Sill,  Oklahoma 


522 


IMPACT  OF  INFLATION 


mZlZSim 

1877 

1980 

%  INC 

105  MM 

$33 

$  63 

91% 

155  MM 

$70 

$134 

91% 

8  INCH 

$90 

$301 

234% 

FIGURE  1 

IMPACT  OF  INFLATION 


FUEL 

JULY 

1978 

MAY 

1980 

% 

INCREASE 

MOGAS  ($/GAL) 

$0.45 

$  1.29 

187% 

DIESEL  ($/GAL) 

0.36 

1.29 

258% 

REPAIR  PARTS 

430.00 

603.00 

40% 

(M109  HOWITZER  TRACK  PADS) 

FIGURE  2 

523 


INDIVIDUAL 

SOLDIER 


MOTOR  POOL 
GARRISON 


INDIRECT  FIRE  ENGAGEMENT  SIMULATION 


INVOLVEMENT 

FIGURE  7 


COMPUTE  FIRING  DATA 

FIGURE  3 


526 


SIMULATION  IN  THE  CANADIAN  FORCES 


By 

Commodore  Gordon  L.  Edwards 
Director  General  Military  Plans  and 
Operations,  National  Defence  Headquarters, 
Ottawa,  Ontario 


ABSTRACT 

The  Canadian  Armed  Forces  stress  the  use  of  simulation  to  both  increc«e 
effectiveness  and  realism,  and  reduce  costs  associated  with  military  training. 
An  integrated  approach  to  the  use  of  simulation  across  all  three  Services 
ensures  that  the  maximum  benefit  is  derived  from  new  technology. 


INTRODUCTION 

One  year  ago.  General  i’amsey  Withers, 
Chief  of  the  Defence  Staff  directed  that 
simulation  be  accorded  a  much  higher 
priority  in  the  training  of  the  Canadian 
Armed  Forces.  A  Simulation  Permanent 
Working  Group  was  formed  to  serve  as  a 
focal  point  for  simulation  throughout  the 
Canadian  Forces . 

The  present  policy  in  effect  across 
all  three  Services  directs  that  simulation 
be  incorporated  into  the  training  system 
whenever  savings  in  resources  or  energy 
accrue  while  improving  or  maintaining 
operational  proficiency.  It  further 
directs  that  in  all  new  projects,  simula¬ 
tion  opportunities  be  addressed.  New 
projects  will  also  address  the  energy 
factor  and  stress  conservation  in  all  non¬ 
renewable  resources . 

Since  the  Second  World  War  the 
Canadian  Navy  has,  by  and  large,  sailed 
in  ships  designed  and  built  in  Canada  and 
the  majority  of  Canada's  military  research 
and  development  has  been  directed  towards 
the  maritime  requirement.  Notwithstanding, 
the  Navy  training  philosophy  has  followed 
that  of  the  Royal  Navy  and  many  training 
devices  have  been  purchased  from  United 
Kingdom  sources.  The  new  patrol  frigate 
program  has  given  new  impetus  to  the 
development  of  naval  simulators  that  will 
replace  our  outdated  trainers  and  match 
the  needs  of  the  updated  DDH  280,  and  the 
new  patrol  frigates. 

Due  to  the  close  association  among 
North  American  aviation  industries  and 
joint  North  American  Air  Defence  agreements 
our  Air  Force  has  adopted  similar  aircraft 
and  consequently  has  similar  requirements 
for  simulators  and  training  devices  as 
the  USAF.  The  recent  additions  to  the 
CF  140  Long  Range  Patrol  Aircraft  and  the 
decision  to  purchase  the  F18  with  their 
new  technologies  have  generated  the  need 
to  push  the  state-of-the-art  to  develop 
associated  modern  training  equipments  and 
methods. 


In  the  Aimy  most  equipment  is  pur¬ 
chased  outside  of  Canada,  generally  in 
the  US,  while  training  and  organizations 
have  tended  to  follow  the  British  Army. 
While  individual  training  standards  have 
been  maintained,  attitudes  towards  the 
use  of  sophisticated  simulation  have  been 
rather  conservative  until  fairly  recently, 
when,  with  the  acquisition  of  the  Leopard 
Tank,  trainers  became  more  aware  of  the 
potential  offered  by  simulation.  This, 
plus  the  high  ammunition  costs  along  with 
large  personnel  turnovers  have  given  those 
responsible  for  training  the  desire  to 
seek  out  new  and  innovative  approaches 
for  maintaining  the  high  standard  for 
which  the  Canadian  soldier  is  renowned. 

While  we  have  a  small  Armed  Forces, 
our  current  inventory  of  simulators  is 
worth  about  $200  million  and  if  tentative 
plans  are  approved  the  value  of  that 
inventory  will  exceed  one  billion  dollars 
by  the  end  of  this  decade.  To  give  you 
an  idea  of  what  we  have  and  our  plans  for 
the  near  future  let  me  run  through  some 
examples . 

COMPUTER  ASSISTED  LEARNING  (CAL) 

In  co-operation  with  the  National 
Research  Council  of  Canada  the  Canadian 
Forces  is  engaged  in  a  five-year  RSD 
project  to  experiment  with  Computer 
Assisted  Learning  with  a  view  to  large- 
scale  implementation  in  our  training 
system.  NFC  has,  in  conjunction  with 
this,  developed  a  high  level,  second 
generation  courseware  Authoring  Language 
known  as  NATAL,  National  Author  Language. 
This  common  instructional  language  is 
easily  adaptable,  transportable,  flexible, 
bilingual  and  is  machine  independent. 

Most  important  it  is  easy  to  use  and 
trials  have  confirmed  that  instructors 
without  previous  ADP  experience  can  pro¬ 
gram  courseware  within  two  weeks.  The 
CAL  project  will  investigate  a  number  of 
potential  uses,  for  example;  substantial 
savings  may  be  expected  in  the  number  of 
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operational  equipments  required  for 
training  purposes. 

NAVY 

Navy  Bridge  Trainer 

Installed  at  Esquimalt  (near 
Victoria,  B.C.)  in  1980,  this  British 
designed  (SOLARTRON)  simulator  effectively 
teaches  junior  officers  in  ship  handling 
techniques,  use  of  radar,  voice  procedure 
and  bridgemanship  leading  to  watch  keeping 
certification.  It  has  a  digital  coastline 
radar  simulation  and  most  significantly 
teaches  the  handling  of  ships  in  close 
company  and  collision  avoidance  under 
various  weather  and  sea  states  as  well  as 
navigation  under  zero  visibility 
conditions.  Needless  to  say  we  dare  not 
afford  to  let  junior  officers  thrash 
around  congested  and  hazardous  waters 
trying  their  hand  at  ramming,  colliding 
and  running  aground,  normally  the  prero¬ 
gative  of  senior  officers.  A  positive 
benefit  of  this  trainer  is  the  greatly 
reduced  time  it  takes  to  reach  watch 
keeping  certificate  standards.  In 
addition,  we  believe  that  it  will  prove 
to  be  effective  in  improving  confidence 
and  morale  of  junior  Naval  officers. 

The  Action  Speed  Tactical  Trainer 

Just  inst  lied  this  summer  at  our 
Maritime  Warfare  School  in  Halifax,  this 
trainer,  designed  and  manufactured  by 
Ferranti  Digital  Systems  UK,  employs 
state-of-the-art  simulation  and  displays 
to  provide  tactical  scenarios  for  exer¬ 
cising  surface,  sub-surface  and  aircraft 
under  various  Naval  combat  conditions. 

Its  purpose  is  to  practise  maritime 
officers  in  tactical  decision-making. 
Capabilities  include  the  full  complement 
of  modern  sensors  and  weapons  systems 
with  the  capability  to  accommodate  future 
developments.  This  trainer  permits  the 
training  cf  officers  to  cope  with  the 
multi-threat  environment  as  well  as  the 
development  and  evaluation  of  new  tactics. 
It  also  provides  a  depth  of  experience 
that  could  not  be  achieved  in  other  than 
a  synthetic  environment.  In  addition  to 
these  recent  acquisitions  we  have  in 
service  on  both  coasts  a  wide  range  of 
sonar,  EW,  radar,  blind  pilotage  and 
operations  trainers,  most  of  which  are 
out  of  date. 

We  are  about  to  start  a  complete 
modernization  program  of  all  Navy  combat 
procedures  training  facilities.  Already 
identified  are  an  advanced  Command  Team 
Trainer,  a  basic  Command  Team  Refresher 
Trainer,  two  Tactical  Data  System 
Trainers,  two  Sonar  Trainers,  and  an  EW 
Trainer,  Yet  to  be  defined  are  a  series 
of  trainers  in  support  of  the  new 
Canadian  patrol  frigate.  In  addition, 
there  is  a  program  to  refurbish  the  entire 
spectrum  of  training  aids  and  devices 
currently  in  the  Navy  inventory. 


ARMY 

Observed  Fire  Simulators 

We  recently  installed  six  Invertron, 
UK  designed  artillery  Observed  Fire  Simu¬ 
lators.  It  is  a  second  generation 
simulator  that  has  proven  to  be  extremely 
effective  in  training  indirect  fire 
observers.  Users  are  very  positive  about 
its  value  and  during  the  first  year  of 
use  at  the  Artillery  School  in  Gagetown, 
New  Brunswick,  ammunition  savings  of 
$1.3M  were  made. 

Leopard  Tank  Training  Devices 

With  the  acquisition  of  the  Leopard 
Tank,  trainers  began  to  realize  the 
importance  of  cost  effective  modern 
training  devices.  Currently  we  have: 

1.  Drivers  Instructional  Cab 
(German) ; 

2.  Turret  Classroom  Instructional 
Models ;  and 

3.  TALAFIT  (Tank,  Laying,  Aiming 
and  Firing  Trainer)  (Belgium) . 

We  are  in  the  process  of  improving 
the  armoured  training  system  to  include 
gunnery  crew  simulators,  tactical  simula¬ 
tors  and  driver  simulators.  One  concept 
is  the  TICS  (Turret  Interaction  Crew 
Simulator) .  This  is  similar  to  the  UCOFT 
(Unit  Conduct  of  Fire  Trainer  for  the  US 
Battle  Tank  Ml) . 

Staff  Training  Tactical  Simulator 

Although  no  acquisition  project  is 
currently  underway  for  an  Army  Staff 
Tactical  Training  Simulator,  we  have 
completed  development  of  a  system  that 
economically  solves  the  displayed  map, 
line  of  sight  and  movement  problems  on  an 
automated  battle  board.  We  are  now  in  a 
position  where  we  can  build  on  this  capa¬ 
bility  to  provide  automated  command  and 
staff  training  from  unit  to  corps  level. 

AIR  FORCE 

Long  Range  Patrol  Aircraft  -  CP  140  Aurora 

By  combining  the  better  features  of 
the  USN  PS  and  S3  trainers  along  with  our 
own  ideas  utilizing  the  state-of-the-art, 
we  have  produced  a  package  that  has  proven 
to  entirely  satisfy  our  training  require¬ 
ments.  Trainers  include: 

1.  The  Flight  Deck  Simulator, 
designed  and  manufactured 
by  CAE  Industries; 

2.  The  Operational  Mission 
Simulator;  and 

3.  The  Integrated  Avionics  Trainer, 
and  six  other  part  task  trainers 
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which  represent  major  sub 
systems  for  maintenance 
training  purposes. 

These  trainers/simulators  have  been  in 
service  for  one  year  and  both  trainers 
and  trainees  are  extremely  enthusiastic. 

CC130  Hercules 

Approved  for  procurement  is  an  up- 
to-date  CC130  Operational  Flight  Trainer 
which  will  include  a  state-of-the-art 
digital  computer  generated  visual  system. 
As  well,  we  are  in  the  process  of 
replacing  six  analogue  basic  jet  trainer 
simulators  with  state-of-the-art  visual 
computer  generated  systems.  We  are  also 
preparing  the  documentation  necessary  for 
approval  to  procure  a  tactical  helicopter 
simulator. 


Sea  King  Helicopter 


We  have  completed  the  replacement  of 
the  analogue  tactics  portion  of  the  Sea 
King  Helicopter  and  are  in  the  process  of 
similarly  replacing  the  analogue  flight 
and  engine  simulators  with  state-of-the- 
art  systems  and  will  be  procuring  entirely 
new  simulators  when  the  Sea  King  is 
replaced . 


CF18  Hornet 

In  support  of  the  new  CF18  aircraft 
we  have  a  contract  with  CAE  Ltd  for  two 
Operations  and  Flight  Tactic  (OFTT) 
trainers  which  include  state-of-the-art 
computer  generated  systems  and  a  weapons 
systems  trainer  in  which  the  basic  OFTT 
will  be  augmented  by  the  latest  air  com¬ 
bat  manoeuvring  systems.  In  addition, 
the  project  includes  the  procurement  of 
two  HOTAS  (Hands  on  Throttle  and  Stick) 
trainer,  a  kit  of  maintenance  trainers 
and  a  host  of  Computer  Assisted  Learning 
(CAL)  stations . 


E3A  AWAC 

DND  in  conjunction  with  Canadian 
Industry  has  been  responsible  for  the 
design,  development  and  manufacture  of  an 
E3A  (AWAC's)  Operational  Flight  Trainer 
and  this  trainer  is  in  the  final  stages 
of  acceptance  testing;  to  be  installed  in 
Germany  for  NATO  AWAC's  aircrews. 


CONCLUSION 


As  you  can  see  the  Canadian  Armed 
Forces  is  dedicated  to  the  philosophy  of 
simulation  as  a  means  of  improving 
training  while  creating  a  hedge  against 
rising  training  costs  and  energy  con¬ 
servation  . 


In  return  for  the  generous  way  you 
have  shared  information  with  ub  and  in 
hosting  many  visitors  at  your  various 
training  establishments,  I  would  like  to 


extend  a  warm  invitation  to  you  to  visit 
any  of  our  training  bases  and  see  any  of 
these,  or  other  training  devices  in 
operation.  We  of  course  would  welcome 
any  joint  development  suggestions  and 
perhaps  at  some  time  should  consider  some 
sort  of  body  where  matters  related  to 
training  development/simulation,  c.r,  can 
be  mutually  shared. 
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ABSTRACT 

Now  more  than  ever  the  U.S.  Army  must  be  prepared  to  survive  and  win  in  battle.  This  paper 
discusses  this  necessity  and  how  the  Army  Is  meeting  this  need  by  the  development  of  the 
National  Training  Center.  An  overview  Is  presented  on  the  training,  Evaluation,  and  control 
concepts  of  the  National  training  Center  and  how  these  concepts  have  been  Implemented  by  the  NTC 
Instrumentation  System. 


BACKGROUND 

The  Need  -  The  Necessity  To  Be  Ready  To  Win 

"The  Army's  primary  objective  Is  to  win  the 
land  battle  -  to  fight  and  win  In  battles  large  or 
small ,  against  whatever  foe,  wherever  we  may  be 
sent  to  war..."  "We  must  assume  the  enemy  we  face 
will  possess  weapons  generally  as  effective  as  our 
own.  And  we  must  calculate  that  he  will  have  them 
in  greater  numbers  than  we  will  be  able  to  deploy, 
at  least  In  the  opening  stages  of  a  conflict." 
"...We  can  expect  very  high  losses  to  occur  In 
short  periods  of  time.  Entire  forces  could  be 
destroyed  quickly  If  they  are  Improperly  em¬ 
ployed."* 

In  order  to  accomplish  Its  objective,  the  U.S. 
Army  must  train  In  peacetime  as  It  will  fight  in 
war.  Terrain  utilization  and  and  weapons  employ¬ 
ment  are  skills  that  the  Army's  leaders  must  have 
honed  to  perfection  through  training.  Maneuver  and 
support  units  must  be  able  to  move,  emplace  and 


‘Operations  FM  100-5  (Washington,  DC,  HQOA,  1  July 
1976)  p.  1-1 


shoot  quickly.  Night  operations  must  he  as  famil¬ 
iar  as  day  operations.  All  units  must  he  able  to 
survive  and  accomplish  their  missions  In  the  hos¬ 
tile  nuclear,  biological  and  chemical  environments 
as  well  as  that  of  electronic  warfare.  Only 
through  peacetime  training  can  these  skills  be 
acquired  so  that  the  confidence  to  deal  with  the 
heightened  challenges  of  war  may  be  developed. 

The  Army  has  structured  Its  training  to  meet 
this  challenge.  The  ARTEP  provides  a  collective 
training  program  for  experiential  training  based  on 
a  TRAIN-EVALUATE-TRAIN  model.  Yet  virtually  every¬ 
where  It  Is  stationed,  the  U.S.  Army  Is  hard 
pressed  to  provide  complete  resources  for  such 
training.  Modern  weapon  systems  have  changed  the 
tempo,  lethality  and  size  of  battle  areas.  The 
same  land  area  which  was  once  ample  for  training 
divisions  Is  now  Inadequate  for  exercising  bri¬ 
gades.  As  Army  units  train  for  war  maneuver  areas 
which  are  now  too  small  constrain  the  realism  of 
their  training.  Manning  of  a  realistic  opposing 
force  and  an  effective  control  and  assessment 
structure  Is,  for  battalion  level  exercises,  beyond 
typical  resources. 
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Civilian  communities  adjacent  to  Installations  replay  the  aerial  combat  for  the  participants  has 

limit  the  Army's  ability  to  fully  utilize  elec-  created  the  type  of  experiential  learning  that  can 

tronlc  warfare  and  close  air  support  consistent  approximate  air-to-air  combat  and  Increase  the 

with  a  realistic  battlefield  environment.  capability  of  U.S.  pilots  to  survive  and  be  suc¬ 

cessful  In  the  early  phases  of  combat. 

The  foregoing  realities  and  technological 

advances  have  laid  the  foundation  fcr  conducting  The  Army  at  the  Training  and  Doctrine  Command 

realistic  large  unit  training  In  a  cost-effective  Combined  Arms  Test  Activity.  Combat  Developments 

manner  to  help  meet  the  current  and  future  Army  Experimentation  Center,  and  other  test  ranges  has 

training  needs.  used  modern  simulation  technology  to  attempt  to 

create  realistic  combat  conditions,  not  with  the 
Historically,  as  combat  units  first  take  up  goal  of  training  personnel  and  units  but  of  testing 

their  role  In  a  war,  the  losses  of  those  units  have  concepts  and  equipment  under  conditions  as  near 

been  very  high,  compared  to  the  losses  the  same  combat  as  possible.  These  agencies  have  taken 

units  suffer  later  In  combat.  Intuitively  one  advantage  of  the  capabilities  of  the  laser,  the 

would  expect  a  combat-experienced  soldier  or  unit  computer,  radar  and  other  technologies  to  reach 

to  suffer  fewer  casualties  and  to  be  more  effective  levels  of  accuracy  and  realism  that  had  not  before 

than  those  without  such  experience.  Certainly,  been  possible.  These  tests,  however,  were  designed 

officers  and  men  of  all  services  can  look  back  at  primarily  to  provide  the  realism  for  the  equipment 

their  participation  In  World  War  II,  Korea,  and  or  mechanical  function  and  were  limited  to  small 

Viet  Nam  to  find  validation  for  such  Intuition.  units.  The  training  for  combat  survival  of  the 

Moreover,  the  statistics  of  units  In  those  wars  Individuals  Involved  during  these  experiments  has 

bear  out  the  conclusion  that  combat  experience  been  of  secondary  Importance. 

Improves  both  the  survivability  and  the  performance 

of  units.  The  Army  Is  now  using  laser  simulations  for 

direct  fire  weapons  to  Increase  the  realism  In 
Over  the  years,  mcny  attempts  have  been  made  training.  For  the  first  time,  opposing  forces  can 
to  realistically  duplicate  the  combat  experience  realistically  engage  each  other  In  mock  combat, 

during  training  when  the  cost  of  learning  Is  meas-  Inflicting  simulated  casualties  and  damage  as  a 

ured  In  dollars  rather  than  In  lives.  Infiltration  direct  result  of  the  engagement  and  not  from  the 

courses  with  live  machine  gun  fire  only  a  few  feet  use  of  slow  and  arbitrary  means  (i.e.,  judgemental 

above  the  ground  and  the  use  of  live  artillery  neclslons  by  umpires), 

fires,  live  close  air  support  and  naval  gunfire 

during  exercises  nave  been  employed.  How  NTC  Meets  The  Need 

No  training  can  comoletely  capture  the  danger  In  a  statement  before  the  House  Appropriations 

-  and  the  associated  fear  -  that  Is  generated  by  Committee,  General  Rogers,  former  Army  Chief  of 

real  combat.  However,  each  of  the  techniques  used  Staff,  made  the  point  that  “the  worst  thing  that 

In  the  past  has  been  Intended  to  provide  some  part  can  happen  Is  for  a  soldier  and  a  unit  to  find 

of  the  sounds  and  feel  of  combat  so  that  In  the  themselves  on  a  battlefield,  fighting  In  anger  for 

future,  the  soldier,  sailor,  or  airman  entering  the  first  time  and  never  to  have  experienced  any- 

combat  would  not  find  It  completely  new  and  foreign  thing  like  It  before." 

to  him. 

Recognizing  the  Increasing  Inadequacy  of 
A  major  contribution  to  the  technological  training  facilities,  the  Secretary  of  the  Army  and 

advancement  of  training  has  been  the  Air  Force  RED  the  Chief  of  Staff  stated  In  their  1978  joint  pos- 

FlAG  activity  at  Nellis  Air  Force  Base,  Here  the  ture  statement  to  Congress  that: 

combination  of  carefully  trained  and  equipped 
"enemy"  aircraft,  a  highly  sophisticated  and  In¬ 
strumented  airspace,  and  the  ability  to  capture  and 
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"The  Army  foresees  one  or  more  National 
Training  Centers,  large  military  reserva¬ 
tions  which  can  support  the  kind  of  com¬ 
bined  arms  training  needed  to  ready  the 
total  Army  for  battle  In  Europe." 

The  National  Training  Center  (NTC)  Is  a  fa¬ 
cility  where  highly  realistic,  comprehensive  and 
Intensified  training  will  be  conducted.  Troops 
will  be  transported  to  this  facility  periodically 
to  become  proficient  In  critical  tasks  that  cannot 
be  accomplished  at  home  stations.  Since  combat 
conditions  are  duplicated  with  great  fidelity  at 
the  NTC,  It  can  also  be  used  as  a  combat  proving 
ground.  Battle  realism,  evaluation  and  feedback  In 
this  environment  require  engagement  measurement  and 
monitoring  and  control  Instrumentation  coupled  with 
data  processing  and  display  capabilities  to  provide 
objective  assessments  and  analyses  of  unit  perform¬ 
ance  with  sufficient  detail  and  timeliness  to 
ensure  maximum  learning.  A  beneficial  by-product 
of  such  Instrumentation  Is  the  ability  to  answer 
critical  questions  of  aggregate  force  readiness 
trends  and  effectiveness  of  doctrine,  organiza¬ 
tions,  equipment  and  training  techniques. 

The  NTC,  with  Its  mixture  of  various  forms  of 
Instrumentation  and  simulations,  and  with  Its 
ability  to  capture  data  for  later  use,  offers  a 
training  environment  that  closely  resembles  real 
combat  while  concommltantly  providing  the  ability 
to  rapidly  analyze  the  events  that  occur  during  the 
exercise,  to  evaluate  the  general  state  of  a  unit's 
training,  to  assess  their  deficiencies  and  to 
provide  In-depth  near-real-time  feedback. 

It  Is  this  concept  of  using  modern  technology 
available  to  train  units  by  creating  a  near  combat 
environment  that  makes  the  NTC  unique.  The  NTC 
concept  Is  to  train  so  realistically  that  the 
soldier's  reactions  to  combat  are  learned  so  well 
he  will  act  Intuitively  and  decisively  In  real 
combat.  This  process,  known  as  experiential 
learning,  Is  the  goal  of  the  NTC. 

The  Training  and  Doctrine  Command,  In  Its 
analysis  of  the  NTC  training  environment,  has 
developed  an  array  of  seven  significant  training 
elements  to  be  stressed  In  the  NTC  development: 


•  Battalion  Task  Force  (BnTFl.  A  combined 
arms  team  with  a  staff  and  a  critical  task 
of  coordinating  combat  power  on  the 
battlefield. 

•  Opposing  Force  (OPFOR1.  A  dedicated  unit, 
sized  and  equipped  to  operate  against  the 
BnTF  In  realistic  numbers,  using  Soviet 
tactics  and  signatures  and  operating  as 
part  of  the  control  force  to  ensure  proper 
balance  In  combat  operations.  This  OPFOR 
must  think  and  act  like  Soviets,  Including 
less  concern  for  casualties  and  more  for 
results. 

•  Electronic  Warfare  (EW).  The  use  of 
jammers  against  U.S.  communications  and 
electromagnetic  devices  In  a  manner 
expected  of  the  Soviets  In  Europe. 

•  Close  Air  Support  (CAS).  The  opportunity 
to  plan  and  execute  joint  air-ground 
operations. 

•  Live  Fire  (LF).  An  Imaginative  use  of 
portable.  Instrumented  targets  to  provide 
a  realistic  threat,  both  offensively  and 
defensively,  to  the  BnTF  under  conditions 
that  permit  the  BnTF  to  coordinate  and 
control  live  fires. 

•  Weapon  Engagement  Simulation  (ES).  The 
use  of  lasers  and  computers  to  simulate 
fires  on  the  battlefield.  Including  a 
realistic  and  believable  assessment  of 
casualties. 

•  Instrumentation.  The  use  of  sensors, 
computers,  and  data  communications  to  tie 
together  the  whole  NTC  activity  and  to 
capture  that  activity  In  a  manner  which 
permits  effective  training  feedback  and 
assessment. 

Together,  these  elements  provide  a  training 
environment  where  Army  maneuver  battalions  can 
undertake  essential  combined  arms  training  which 
cannot  be  accomplished  at  home  station  due  to 
physical  limitations  and  the  prohibitive  cost  for 
providing  an  NTC  type  environment.  In  addition  to 
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this  training  role,  the  NTC  also  provides  an  envi¬ 
ronment  to  gather  data  about  simulated  battlefield 
performance  and  the  effectiveness  of  U.S.  Army 
organizations,  doctrine,  procedures,  tactics  and 
weapon  systems  under  realistic  simulated  combat 
conditions. 

NTC  PHASE  I  CONCEPT  -  AN  OVERVIEW 

The  National  Training  Center  (NTC)  Is  being 
Implemented  In  Phases  of  which  Phases  I  and  II  have 
been  defined.  Phase  I,  when  Implemented,  will 
provide  a  place  where  Army  units  can  undertake 
essential  combined  arms  training  that  cannot  be 
done  at  home  stations  because  of  limitations,  or 
the  prohibitive  total  cost  of  providing  a  NTC  Phase 
I  type  environment  at  all  or  selected  division 
Installations. 

The  Training  Environment  and  Concept  of  NTC 

The  elements  of  the  training  environment  for 
the  NTC  Phase  I  Include: 

•  Battalion  Task  Force  (BnTF).  The  basic 
building  block  of  mounted  warfare  Is  and 
will  be  the  cross-reinforced  tank  or 
mechanized  company  team  or  battalion  task 
force.  After  corps  and  division  com¬ 
manders  have  set  In  motion  the  necessary 
concentration  of  power,  they  must  turn 
over  the  Immediate  direction  of  the  battle 
to  brigade  and  battalion  commanders.  The 
brigade  and  battalion  commanders  then  must 
fit  the  forces  to  the  ground,  maneuver 
against  the  enemy  as  the  battle  develops, 
and  coordinate  the  concentration  of  fire¬ 
power.  The  battalion  task  force  will  be 
the  element  that  the  NTC  training  environ¬ 
ment  Is  designed  to  train  as  the  Initial 
goals  of  Phase  I.  Initially,  one  battal¬ 
ion  engagement  exercise  will  be  conducted 
at  a  time.  Later  In  Phase  I,  expanded 
Instrumentation  capabilities  will  allow 
two  simultaneous  battalion  level  exer¬ 
cises. 

e  Opposing  Force  (OPFOR).  A  realistic 
"enemy"  force  will  be  provided  by  a 
dedicated  unit,  sized  to  achieve  force 

533 


ratios  of  at  least  3  to  1  against  a  U.S. 
battalion  during  defense  missions.  The 
OPFOR  will  use  Soviet  tactics  and  signa¬ 
tures,  with  the  vehicles  and  weapons 
capabilities  of  Soviet  type  motorized 
rifle  regiment  replicated.  The  OPFOR  will 
consist  of  approximately  1000  men  and  230 
tracked  vehicles  equipped  with  silhouette 
replicas  (VISMOD  kits). 

•  Electronic  Warfare  (EW).  Jammers  will 
replicate  ground  based  threat  communica¬ 
tion  jammers,  capable  of  1.5  kW  opera¬ 
tions,  on  a  narrow  electromagnetic  band 
targeted  against  those  '•ommunlcatlons  nets 
found  In  battalion  and  brigade.  Addi¬ 
tional  equipment,  to  be  suppllec  by  the 
Air  Force,  will  duplicate  a  representative 
air  defense  threat  In  the  vicinity  of  the 
FEBA. 

•  Close  Air  Support  (CAS).  This  will  be 
provided  from  George  AFB  and  RED  FLAG 
operations  at  Nellis  AFB.  It  may  also 
encompass  Marine  and  Navy  air  support  from 
Twenty  Nine  Palms  or  China  Lake.  This  NTC 
Phase  I  will  offer  opportunities  to  prac¬ 
tice  the  joint  planning  necessary  for 
air-ground  operations  at  the  battalion 
task  force  level  with  a  real-time  casualty 
assessment  system  supported  by  Instru¬ 
mentation. 

•  Live  Fire  (LF).  This  will  be  conducted  at 
realistic  engagement  ranges  and  target 
arrays,  with  all  elements  of  direct  and 
indirect  fire  integrated.  Minimum  re¬ 
strictions  commensurate  with  safety  will 
be  applied.  Targets  will  be  portable  and 
have  sufficient  Instrumentation  to  provide 
Input  for  after  action  review.  Targets 
will  respond  to  live  fire  hits  or  laser 
fire  from  DRAGONS  and  TOWS. 

•  Engagement  Simulation  (ES).  The  MILES 
system  will  be  used  down  to  Individual 
weapon  level.  Engagement  simulation  data 
provided  by  MILES  will  be  reported  by 
telemetry  on  selected  weapons. 


•  Instrumentation.  The  NTC  Instrumentation 
components  Include  time-space  position 
location,  key  event  recording,  voice  and 
video  recording,  and  appropriate  analysis 
and  playback  facilities. 

Training  at  the  NTC  Phase  I  Mill  be  based  upon 
the  TRAIN-EVAUJATE-TRAIN  model  with  detailed  feed¬ 
back  after  each  mission  and  a  final  diagnostic 
After-Actlon-Revlew  (AAR)  and  Take-Home-Package  to 
serve  as  a  basis  to  guide  subsequent  home  station 
training  programs.  The  training  Mill  focus  on  Im¬ 
proving  performance  In  the  five  levels  of  activity 
Mlthln  the  battalion  task  force:  execution,  con¬ 
trol,  coordination,  support  and  planning. 

The  NTC  Phase  I  training  should  be  vleMed  as 
an  extension  of  and  building  on  home  station 
training  Mlth  heightened  realism  In  battlefield 
conditions  and  objective  standards.  The  NTC  Phase 
I  Mill  concentrate  on  minimum  constraint  operations 
at  the  combined  arms  task  force  level  and  will  pro¬ 
vide  comnand  and  staff  exercise  opportunities  at 
the  brigade  level. 

The  NIC  training  scenarios  will  be  tailored 
for  each  unit  based  on: 

e  Current  status  of  collective  training  - 
proficiency  based  on  ARTEP, 

e  Resources  at  home  station, 

•  Contingency  plans, 

e  Training  goals,  and 

•  Task  organization. 

The  training  experience  Mill  be  based  upon  a 
European  analogue.  Units  Mill  exercise  emergency 
deployment  plans;  move  to  and  draw  POMCUS-Hke 
prepositioned  equipment;  deploy  to  field  positions 
similar  to  European  defense  plans;  and  execute 
scenarios  of  appropriate  tactical  missions.  By 
1984  brigade  headquarters  Mill  be  responsible  for 
battle  management  of  a  mix  of  one  real  and  up  to 
two  notional  battalions.  Battalion  commanders  and 
their  staffs  will  be  exercised  In  a  realistic 
command  environment  using  CPX  role-playing  tech¬ 
niques  similar  to  those  used  In  the  Combined  Arms 
Tactical  Training  Simulator  (CATTS).  Each  CONUS 
armor/mechanized  battalion  conmander  and  staff  will 


then  train  at  Fort  Irwin  twice  each  18-month  peri¬ 
od,  once  without  troops  on  the  notional  battle 
facility,  and  once  with  the  entire  battalion. 

Evaluation  Concept 

The  evaluation  function  of  the  TRAIN- 
EVALUATE-TRAIN  model  at  the  NTC  Phase  I  will  con¬ 
centrate  on  the  phases  of  the  command  and  control 
process  at  the  battalion  task  force  level:  col¬ 
lection  of  Information,  planning.  Issuing  orders, 
assessing  and  supervising  the  progress  of  opera¬ 
tions. 

At  the  NTC  Phase  I,  the  primary  target  of 
evaluation  rnd  corrective  training  will  be  the 
battalion's  ability  to  orchestrate  the  application 
of  Its  combat  power.  A  Take-Home-Package  for  home 
station  training  will  provide  the  opportunity  for 
review  and  remedial  training.  The  evaluation 
concept  Is  designed  to  provide: 

a  An  Increase  In  proficiency  of  battalion 
combined  arms  teams  as  they  execute  a 
series  of  missions. 

•  A  comprehensive  assessment  of  unit  train¬ 
ing  to  serve  as  a  basis  for  subsequent 
home  station  training, 

The  evaluation  concept  will  be  based  on  a 
determination  of  what  should  be  collected  and 
measured;  what  will  be  the  Information  feedback 
levels,  critique  audiences  and  frequency;  and  what 
the  evaluation  packages  are  used  for. 

Instrumentation  and  Control  Concept 

The  Instrumentation  and  control  system  has 
been  designed  to  support  the  objectives  outlined  In 
the  training  and  evaluation  concepts.  The  system 
will  collect  and  report  data  during  performance  of 
unit  tasks;  enhance  overall  realism  and  control  of 
the  exercise;  record  and  process  collected  data, 
and  provide  computer  generated  Imagery  or  graphic 
displays  for  assessment  and  control  of  the  exer¬ 
cise.  Data  collection  and  processing  will  be 
objective  and  primarily  based  upon  Instrumentation. 
Some  actions  within  the  levels  of  activity  upon 
which  the  training  end  evaluation  will  be  focused 
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are  not  suitable  for  direct  Instrumented  collec¬ 
tion,  These  shall  be  collected  and  processed 
relying  on  controllers.  The  Instrumentation  system 
will,  however,  be  structured  to  support  and  ex¬ 
pedite  these  controller  Inputs. 

The  design  of  the  Instrumentation  system  and 
the  mix  of  Instrumented  and  controller  functions 
will  be  based  on  the  system's  ability  to  support 
the  TRAIN-EVALUATE-TRA1N  model.  The  analysis  of 
the  evaluation/instrumentation  system's  ability  to 
support  the  NTC  Phase  I  will  be  built  around  four 
basic  Issues: 

e  What  should  be  collected  and  measured? 

e  What  can  be  collected  and  measured  and  at 
what  level? 

e  What  are  the  methods  and  frequency  of 
feedback  presentation? 

e  What  will  the  data  be  used  for? 

The  Instrumentation  system,  collection  methods 
and  playback  techniques  are  described  In  the  NTC 
Development  Plan  published  by  the  Army  Training  and 
Doctrine  Command. 

The  Training  Mlss^n  and  Its  Implementation 

The  unique  aspect  of  a  National  Training 
Center  (NTC)  Is  the  total  emersion  of  a  battalion 
task  force  In  a  realistic  battle  environment.  The 
NTC  training  will  be  as  realistic  as  technology  and 
safety  will  permit.  Equally  Important,  an  Instru¬ 
mented  NTC  will  allow  the  transparent  collection  of 
hard  analytical  data  from  which  objective  battle¬ 
field  performance  effectiveness  can  be  derived. 

Training  Objectives 

The  NTC  training  objectives  are  designed  to 
fill  the  gap  between  home  stations  unit  training 
and  combat.  They  are  Intended  to  serve  as  a  basis 
for  NTC  training  and  evaluations.  These  specific 
objectives  are: 

e  Increase  the  proficiency  of  brigade  and 
battalion  commanders  and  staffs  to  plan 
and  exercise  command  and  control  of  their 
forces  while  executing  combined  arms 


tactical  missions  In  a  realistic  threat 
environment. 

a  Increase  the  proficiency  of  brigade  and 
battalion  task  forces  to  coordinate  and 
apply  all  types  of  fire  and  terrain  mod¬ 
ification  to  Increase  force  effectiveness 
and  decrease  unit  vulnerability. 

a  Increase  the  proficiency  of  the  battalion 
task  force  to  employ  and  coordinate 
organic  and  supporting  direct  and  Indirect 
fires,  combat  support  and  combat  service 
support  systems  against  a  realistic 
opposing  force  in  a  free  play  exercise. 

e  Increase  the  tactical  proficiency  of  the 
maneuver  elements  and  leaders  within  a 
battalion  task  force  In  the  conduct  of 
their  combat  missions  in  a  realistic 
hostile  environment,  to  Include:  (1)  use 
of  tactics,  terrain  and  terrain  modifica¬ 
tion  to  maximize  combat  power  at  critical 
times  and  places;  (2)  employ  unit  gunnery 
skills  In  conjunction  with  supporting 
direct  and  indirect  fires  In  a  near-real - 
combat  environment  to  acquire  and  service 
targets  representative  of  an  appropriate 
threat  force  at  realistic  ranges  and 
dispersion;  (3)  Increase  soldier  and 
leader  confidence  in  their  proficiency  of 
maneuvering  company  team  and  higher  units 
in  close  proximity  to  live  fire;  (4) 
exercise  EU  and  NBC  counter  measures 
necessary  to  survive  and  function  on  a 
"dirty"  battlefield;  and  (b)  Integrate  air 
cavalry  and  attack  helicopters  Into  the 
scheme  of  maneuver  and  to  perfect  the 
control  and  coordination  techniques  unique 
to  those  resources. 

•  Create  an  opportunity  for  brigades  and 
battalions  to  plan  and  execute  realistic 
combat  service  support  operations. 

In  addition  to  the  training  objectives  dis¬ 
cussed  above,  the  NTC  will  also  meet  the  following 
evaluation  objectives: 
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•  Provide  a  measure  of  the  Increase  In 
combat  effectiveness  of  all  elements  of 
combined  arms  task  forces  achieved  during 
training  at  the  NTC. 

e  Provide  real-time  player  feedback  which 
makes  possible  the  compilation  of  an 
accurate  objective  record  of  tactical 
performance  that  can  be  used  in  After- 
Action-Reviews  (AARs)  for  all  levels 
during  the  NTC  training  period  and  after 
units  return  to  home  station. 

e  Assist  In  the  evaluation  of  present  tac¬ 
tical  doctrine,  tables  of  organization  and 
equipment,  and  training  doctrine  at  the 
brigade  and  battalion  level. 

Units  trained  at  the  National  Training  Center 
(NTC)  will  be  required  to  perform  a  large  number  of 
defensive  and  offensive  tactical  missions.  These 
tactical  missions  arc  listed  below. 

e  Movement  to  contact 

e  Hasty  attack 

e  Deliberate  attack 

e  Defend  In  sector 

e  Defend  from  a  battle  area 

•  Hasty  defense 

•  Delay  In  sector 

•  D1 sengagement 

•  Counterattack 

•  Defend  a  battle  position 

•  Deliberate  defense 

•  Reconnaissance  and  security 

•  Create  and  defend  a  strong  point 

A  series  of  tactical  mission  analyses  and 
support  packages  will  be  prepared  for  each  of  the 
above  missions  and  will  be  used  to  conduct  and 
control  the  exercises. 

NTC  PHASE  II  -  PRELIMINARY  OVERVIEW 

Building  on  the  experiences  of  Phase  I  It  Is 
anticipated  that  Phase  II  will  move  beyond  Phase  I 
as  the  state  of  the  art  in  training  and  Instru¬ 
mentation  moves  to  meet  the  NTC  requirements.  It 
Is  expected  that  Phase  II  will  Include: 


•  Increase  In  the  number  of  player  units 
that  can  be  monitored  and  controlled. 

e  Upgrading  of  the  MILES  system  to  Include 
new  weapons  and  target  acquisition  capa¬ 
bilities. 

e  Improved  techniques  and  Instrumentation  to 
address  Indirect  fire,  air  defense  and 
U.S.  Air  Force  close  air  support  simula¬ 
tion. 

e  Improved  After-Action-Review  (AAR)  tech¬ 
niques  to  Include  development  of  special 
presentation  methods. 

Similar  to  Phase  I,  NTC  Phase  II  will  provide 
an  order  of  magnitude  increase  In  the  proficiency 
of  the  Army's  tactical  forces  through  effective  and 
realistic  training.  It  will  provide  the  experi¬ 
ential  base  which  will  allow  our  forces  to  fight 
outnumbered  and  win  the  first  and  subsequent 
battles  of  any  conflict. 

NTC  INSTRUMENTATION  SYSTEM  OVERVIEW 
Key  Requirements  -  The  Design  Drivers 

Out  of  the  NTC  training  concept  which  drove 
the  evolving  structure  of  the  NTC  Instrumentation 
System  a  number  of  Imperative  requirements  became 
the  bases  for  the  hardware  and  software  cesigns: 

•  Future  growth  must  be  supported  without 
causing  significant  redesign  and  recoding 
of  software; 

•  Instrumentation  must  be  as  transparent  as 
possible; 

•  The  historical  data  collected  must  be 
capable  of  being  recalled  and  portions 
selected  by  training  analysts  during  the 
course  of  the  exercise  for  subsequent 
After-Action-Review  (AAR)  presentations; 

•  Instrumented  data  collection  must  be 
supplemented  with  voice  recordings  of 
radio  nets,  video  picture  input  from  the 
exercise  area,  and  field/observer  con¬ 
troller  Inputs,  and 

•  Technical,  schedule  and  cost  risks  will  be 
minimized. 
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System  Description 


In  order  to  minimize  interface  complexity,  and 
correspondingly  reduce  technical  risks,  a  system 
architecture  has  been  synthesized  that  consists  of 
three  major  subsystems:  (1)  Core  Instrumentation 
Subsystem  (CIS),  (2)  Range  Data  Measurement  Sub¬ 
system  (RDMS) ,  and  (3)  Range  Monitoring  and  Control 
Subsystem  (RMCS).  This  system  architecture  is 
presented  In  Figure  1  along  with  the  allocation  of 
functional  areas  to  all  subsystem  components 
(Figures  1A-1C). 


FIGURE  1  NTC  PHASE  I  INSTRUMENTATION 
ARCHITECTURE 


Range  Data  Measurement  Subsystem  (RDMS) 

The  RDMS  provides  real-time  position  location 
and  engagement  event  data  on  all  Instrumented 
players  In  the  Engagement  Simulation  (ES)  and  Live 
Fire  (LF)  exercises.  As  shown  in  Figure  1A,  the 
RDMS  Is  composed  of  three  major  components:  (1) 
Tracking  and  Communication  Component  (TCC),  (2) 
Computational  Component  (CC),  and  (3)  Player  Unit 
Component  (PUC).  The  RDMS  Player  Unit  Component 
Includes  the  Transponder  Component  (TC)  and  the 
Weapon  Engagement  Simulation  Component  (WESC)  which 
is  the  Army's  MILES  system. 

The  remote,  unmanned  Central  Station  (CS) 
within  the  TCC  selects  the  transmission  path  to 
Player  Units  (PUs),  provides  a  two  way  digital  data 
link  to  Player  Units  (PUs)  and  collects  range  data. 
Event  messages  and  slant  range  data  received  by  the 
Central  Station  (CS)  are  time  tagged  then  relayed 
to  the  CC.  The  CC  computes  PU  position,  decodes 
event  data  for  validity  and  transmits  the  position 
and  event  data  to  the  Core  Instrumentation  Subsys¬ 
tem  (CIS).  Commands  from  the  CIS  are  relayed  to 
the  player  unit  through  the  CC  and  TCC. 


RANGE  DATA 
MEASUREMENT 
SUBSYSTEM 
(RDMS) 


FIGURE  1A  RANGE  DATA  MEASUREMENT  SUBSYSTEM 
(RDMS)  SYSTEM  ARCHITECTURE 


The  Tracking  and  Communications  Component 
(TCC)  consists  of  a  number  of  Micro  A  and  Micro  A/D 
stations  located  at  known  surveyed  points.  The 
Micro  A  and  Micro  A/D  stations  measure  range  to  the 
selected  player  unit  and  In  addition  provide  a 
digital  data  link  between  the  Central  Station  and 
the  Player  Unit.  The  Micro  A/D  station  acts  as  a 
relay  to  Micro  A  stations  that  are  not  in  radio 
line  of  sight  of  the  Central  Station.  The  system 
remains  in  a  quiescent  state  until  a  command  from 
the  Central  Station  triggers  a  ranging  interroga¬ 
tion  command. 

A  Player  Unit  (PU)  includes  a  discretely 
addressable  Micro  B  unit  that  serves  as  a  range 
transponder  and  communications  link.  Additionally, 
a  PU  contains  an  Input/Output  (I/O)  device,  and  a 
Transponder  u..vonent  (TC),  attached  to  a  Weapon 
Engagement  Simulation  Component  (WESC)  unit.  A  PU 
can  receive  and  store  engagement  messages  from  the 
I/O  and  TC/WESC,  and  transmit  the  data  to  the  CS 
upon  TCC  request.  Initial  operational  capability 
will  be  125  player  units  with  expansion  to  500 
player  units  occurring  later  in  Phase  I. 

The  Weapon  Engagement  Simulation  Component 
(WESC)  simulates  direct  fire  weapons  effects  In 
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support  of  a  free  play  engagement  simulation  be¬ 
tween  battalion  TF  OPFOR  elements.  Functions  per¬ 
formed  by  WESC  Include:  simulation  of  direct 

weapon  firing  cues,  computation  of  direct  fire 
casualty  and  damage.  Implementation  of  direct  fire 
effects,  generation  of  firing  and  weapon  effect 
engagement  events  and  hand-off  of  this  data  to  the 
I/O  through  TC  for  ultimate  transmission  to  the 
CIS. 

The  Computational  Component  (CC)  consists  of  a 
multlple-CPU  processor  and  software  necessary  to 
compute  PU  Position  Location  (PL)  data.  Initial 
loading  and  realtime  operations  control  of  the  TCC 
remote,  unmanned  Central  Station  (CS)  Is  accom¬ 
plished  by  the  CC. 

Range  Monitoring  and  Control  Subsystem  (RMCS) 

The  RMCS  provides  the  means  to  monitor  and 
control  all  activities  on  the  NTC  Engagement  Simu¬ 
lation  (ES)  and  Live  Fire  (LF)  ranges.  These 
capabilities  include  automated  and  human  sensors 
and  a  backbone  communications  component  to  tie 
these  sensors  together  and  connect  them  with  the 
CIS. 

As  shown  In  Figur'S  IB,  the  RMCS  consists  of 
six  major  components:  (1)  Range  Comnunlcatlons 

Component  (RCC),  (2)  Spectrum  Analyzer  Component 
(SAC),  (3)  Live  Fire  Component  (LFC),  (4)  Voice  and 
Video  Monitoring  Component  (VVMC),  (5)  Field  Con¬ 
troller  Component  (FCC),  and  (6)  Opposing  Force 
Component  (OFC). 


‘Growth  Option 


FIGURE  IB  RANGE  MONITORING  AND  CONTROL 
SUBSYSTEM  (RMCS)  SYSTEM 
ARCHITECTURE 


The  Range  Communications  Component  (RCC) 
provides  voice,  digital  and  video  communications 
between  RMCS  components  and  the  CIS.  Specifically, 
voice  communications  are  provided  between  the  CIS 
operators  and  their  counterparts  In  the  VVMC,  FTFC, 
FCC  and  OFC  components.  Digital  communications  are 
provided  between  the  IDCC  In  the  CIS  and  the  SAC 
and  LFC  components.  Video  communications  are 
provided  between  the  IDCC  and  VVCEC  components  In 
the  CIS  and  the  VVMC  and  FTFC  components. 

The  Spectrum  Analyzer  Component  (SAC)  provides 
the  means  to  measure,  record  and  transmit  all 
relevant  NTC  EM  emissions  which  may  interfere  with 
other  NTC  or  non-NTC  (i.e.,  GOLDSTONE)  operations. 
The  SAC,  under  control  from  the  CIS,  will  continu¬ 
ously  monitor  an  assigned  frequency  range  at  an 
assigned  rate.  Whenever  an  emission  Is  detected 
which  exceeds  a  threshold  value  set  by  the  CIS,  the 
SAC  will  report  this  occurrence  (time,  frequency 
and  value)  to  the  CIS. 

The  Live  Fire  Component  (LFC)  provides  a 
realistic,  simulated,  combat  environment  for  NTC 
Live  Fire  exercises.  Specifically,  the  LFC  sim¬ 
ulates  a  dynamic  OPFOR  target  array  and  engagement 
scenario;  generates  live  fire  effects  cues  from  the 
OPFOR;  scores  and  records  live  fire  results 
(events);  and  transmits  all  live  fire  event  data  to 
the  CIS. 

The  Voice  and  Video  Monitoring  Component 
(VVMC)  provides  both  fixed  and  mobile  video 
recording  elements  to  record  key  engagement 
simulation  and  live  fire  events.  The  unmanned 
fixed  video  element  will  be  controlled  directly 
from  the  VVCEC  within  the  CIS.  Mobile  video  teams 
will  be  directed  from  the  VVCEC  component  within 
the  CIS  In  response  to  missions  assigned  by  tMC  or 
TAF  operators. 

The  Field  Controller  Component  (FCC)  provides 
nonlntruslve  observation  of  the  battalion  TF  during 
both  Engagement  Simulation  (ES)  and  Live  Fire  (LF) 
exercises.  Specific  functions  performed  by  the  FCC 
include:  enforcement  of  the  rules  of  engagement; 
assessment  of  indirect  fire  casualties;  Implementa¬ 
tion  of  indirect  fire  weapon  effects  cues  (fire 
marking);  assuring  range  safety  and  the  recording 
and  communication  of  battalion  TF  activities  based 
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on  human  observations.  This  component  will  be 
manned  by  Army  personnel . 

The  Opposing  Force  Component  (OFC)  simulates 
the  opposing  force  In  the  free  play  Engagement 
Simulation  (ES)  between  the  battalion  TF  and  the 
OPFOR.  Specific  functions  performed  by  OFC  In¬ 
clude:  the  simulation  of  all  OPFOR  operations  (C3, 
maneuver,  fire,  administration,  log,  etc.);  the 
observation  of  battalion  TF  activities  and  the 
communication  of  these  observations  to  the  CIS 
personnel;  and  execution  of  CIS  specified  OPFOR 
scenarios  to  achieve  the  desired  training  missions 
and  goals.  This  component  Is  also  manned  by  Army 
personnel . 

Core  Instrumentation  Subsystem  (CIS) 

The  CIS  provides  all  real-time  data  processing 
and  interactive  display  capabilities  needed  to 
monitor,  coimand,  and  control  all  NTC  Engagement 
Simulation  (ES)  and  Live  Fire  (LF)  exercise  activi¬ 
ties.  The  CIS  also  provides  data  processing, 
interactive  display,  voice  and  video  editing  and 
training  material  production  capabilities  needed  to 
synthesize  and  present  near-real -time  After-Action- 
Reviews  (AARs),  both  In  the  field  and  In  the  CIS 
AAR  theater,  and  to  produce  take-home  training 
packages.  Finally,  the  CIS  provides  the  data 
processing  and  interactive  display  capabilities 
required  to  support  Training  Developments  (TD)  and 
Combat  Developments  (CD)  research  at  the  NTC. 

As  shown  In  Figure  1C,  the  CIS  consists  of  six 
major  components:  (1)  Digital  Interface  Component 
(DIC),  (2)  Computational  Component  (CC),  (3)  In¬ 
teractive  Display  and  Control  Component  (IDCC),  (4) 
Voice/Video  Control  and  Editing  Component  (VVCEC), 

(5)  Field  Training  Feedback  Component  (FTFC),  and 

(6)  the  Environmental  Protection  Shelter  (EPS). 
The  IDCC  is  further  decomposed  into  four  major 
functional  elements:  (1)  Exercise  Monitoring  and 
Control  (EMC),  (2)  Training  Analysis  and  Feedback 
(TAF),  (3)  Command  Battle  Simulation  (CBS)  and  (4) 
Experimental  Test  Bed  (ETB). 

The  Digital  Interface  Component  (DIC)  provides 
the  single  Input/output  (I/O)  Interface  for  all 
digital  data  communications  between  the  CIS  and  the 
ROMS  and  RMCS  subsystems,  respectively.  The  DIC 


FIGURE  1C  CORE  INSTRUMENTATION  SUBSYSTEM 
(CIS)  ARCHITECTURE 


Implements  digital  communications  protocol  between 
the  CIS  and  external  subsystems.  It  reformats  and 
provides  data  buffering  for  all  CIS  digital  I/O. 
In  short,  the  DIC  centralizes  all  digital  I/O  data 
communications  for  the  CIS  and  preprocesses  these 
data  to  transform  them  Into  the  proper  format 
required  by  the  CIS.  The  DIC  performs  a  similar 
function  for  data  output  from  the  CIS  to  the  ROMS 
and  RMCS  subsystems. 

The  Computational  Component  (CC)  performs  the 
mainline  computation  in  support  of  all  CIS  exercise 
monitoring,  command,  control,  and  training  feedback 
activities.  Specific  computational  processing 
performed  by  the  CC  include:  (1)  state  estimation 
for  all  Instrumented  players  in  the  exercise,  (2) 
real-time  casualty  assessment  for  all  direct  and 
indirect  fire  weapon  engagements  (those  not  per¬ 
formed  or  only  partially  performed  by  the  WESC), 
(3)  real-time  statistical  analyses  to  provide  train¬ 
ing  assessment,  (4)  spectrum  management  analyses, 
and  (5)  range  operations  analyses. 
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The  Interactive  Display  and  Control  Component 
(IDCC)  provides  the  real-time  Interactive  data 
display  and  control  facilities  required  for  CIS 
controllers  to  direct  all  aspects  of  the  NTC  train¬ 
ing  exercise  and  provides  near  real-time  training 
data  feedback.  Specifically,  the  IDCC  provides  a 
digital  background  map,  selectable  tactical  sym¬ 
bology,  engagement  event  data,  statistical  per¬ 
formance  data,  and  function  key,  keyboard  and 
Interactive  menus  to  Interactively  control  all 
aspects  of  the  CIS  subsystem.  As  shown  In  Figure 
1C,  the  IDCC  Implements  the  CIS  display  and  Inter¬ 
active  control  operations  of:  Exercise  Monitoring 
and  Control  (EMC),  Training  Analysis  and  Feedback 
(TAF),  Command  Battle  Simulation  (CBS)  and  Experi¬ 
mental  Test  Bed  (ET8). 

The  Voice/Video  Control  and  Editing  Component 
(VVCEC)  provides  all  facilities  needed  to  record, 
archive,  edit  and  replay  relevant  voice  and  video 
data  obtained  by  monitoring  battalion  TF  and  OPFOR 
field  operations.  Specifically,  the  VVCEC  provides 
the  means  to  record,  edit,  and  replay  BLUEFOR 
tactical  communications.  It  also  provides  an 
Interactive  software  system  to  assist  tactical 
communications  monitors  to  manually  Input  key  COMMO 
event  data.  Finally,  the  VVCEC  provides  similar 
recording,  editing  and  replay  facilities  for  all 
video  data  collected  in  the  field  by  a  fixed  video 
camera  remotely  controlled  from  the  CIS  and  by  five 
mobile  video  cameras  operated  by  the  field  video 
teams  directed  from  within  the  CIS. 

The  Field  Training  Feedback  Component  (FTFC) 
provides  a  self-contained  mobile  display  capability 
to  present  field  After-Action-Reviews  (AARs).  The 
FTFC  will  provide  display  capabilities  similar  to 
those  available  within  the  CIS  AAR  theater  Includ¬ 
ing  a  large  screen  display  and  color  monitors.  The 
FTFC  display  will  be  generated  using  TAF  capabili¬ 
ties  In  the  CIS  under  the  direction  of  Cue  field 
AAR  director  using  voice  communications  provided  by 
the  Range  Communications  Component  (RCC)  within  the 
RMCS. 

The  Environmental  Protection  Shelter  (EPS) 
provides  the  operational  environment  for  all  CIS 
personnel  and  equipment.  Specifically,  the  EPS 
provides  physical  security,  conditioned  power. 


light,  air  conditioning  and  operational  and  main¬ 
tenance  facilities,  equipment  and  supplies. 

Subsystem  Interfaces 

As  discussed  the  NTC  Instrumentation  System 
(NTC-IS)  has  been  divided  Into  three  major  sub¬ 
systems  (CIS,  ROMS ,  and  RMCS)  with  functions 
allocated  to  each  subsystem  to  assure  that  the 
functional  and  physical  Interfaces  between  them  are 
simple  and  straightforward.  The  system  Interface 
diagram  for  the  NTC  Instrumentation  System  (NTC-IS) 
Is  presented  In  Figure  2.  As  shown,  data  generally 
flows  from  the  ROMS  and  RMCS  Into  the  CIS  where  it 
Is  processed  and  displayed  for-  decision  making. 
Control,  on  the  other  hand,  tends  to  flow  from  the 
CIS  to  the  two  other  subsystems  (RDMS  and  RMCS). 

Notice  that  all  digital  data  interfaces  are 
Implemented  by  the  Digital  Interface  Component 
(DIC)  within  the  CIS.  In  this  design,  the  DIC 
provides  all  hardware  and  software  capabilities  to 
assure  data  and  physical  compatabl 1 Ity  between 
subsystems.  That  is,  the  DIC  performs  all  elec¬ 
tronic  signal  and  data  transformations  required  to 
Implement  the  needed  Interface  thereby  decoupling 
all  otiier  data  processing  functions  internal  to  the 
respective  subsystems.  This  design  approach  has 
provided  the  flexibility  required  to  carry  on 
development  of  each  of  the  subsystems  in  parallel 
and  also  provide  for  future  growth. 

The  digital  interface  between  the  RDMS  and  CIS 
has  been  successfully  demonstrated  during  the  NTC 
I -ALPHA  Test  where  the  General  Dynamics  Electronics 
RMS  II  and  Xerox  WESC  systems  were  used  to  Imple¬ 
ment  the  RDMS.  Because  of  this,  all  data  formats 
are  clearly  defined  (based  on  well  known  RDMS 
capabilities)  and  tl<e  physical  Interface  can  be 
Implemented  using  standard  off-the-shelf  inter¬ 
computer  communications  equipments  similar  to  those 
used  in  I -ALPHA. 

CORE  INSTRUMENTATION  SUBSYSTEM  OPERATIONS  OVERVIEW 

The  CIS  functions  as  the  NTC  Exercise  Opera¬ 
tions  Center  during  an  exercise.  Therefore,  an 
operational  overview  Is  useful  In  gaining  an  under¬ 
standing  of  how  the  engagement  will  be  monitored 
and  controlled. 
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The  heart  of  CIS  operations  are  the  functions 
performed  within  Exercise  Monitoring  and  Control 
(EMu)  and  Training  Analysis  and  Feedback  (TAF).  It 
is  therefore  useful  to  explore  the  functions  of  the 
EMC/TAF  controller  stations  and  the  activities 
performed  within  the  EMC/TAF  Operations  Center. 

As  shown  In  Figure  3  the  EMC/TAF  Operations 
Center  Is  located  In  the  Environmental  Protection 
Shelter  (EPS)  wnere  the  data  from  the  Range  Data 
Measurement  Subsystem  (RCMS)  and  the  voice  radio 
and  video  Inputs  Interface  with  the  CIS. 

EMC/TAF  Operations  Center  Layout 

Figure  4  presents  the  layout  of  the  operator 
stations  within  the  EMC/TAF  Operations  Center.  As 
Indicated,  there  are  eight  operator  stations,  each 
assigned  unique  functional  responsibilities. 

STATION  1:  TAF  OPERATIONS 

Operators  at  this  station  are  allocated  the 
responsibility  to  analyze  exercise  data  to  extract 


Important  training  feedback  data  in  order  to  meet 
the  training  objectives  specified  for  each  exercise 
segment.  The  Training  Analysis  and  Feedback 
Officer  (TAFO)  and  his  assistants  structure  an 
After-ActionReview  (AAR)  and  build  material  to  fill 
out  this  AAR  structure  during  an  ongoing  exercise 
segment. 

STATIONS  2,  3,  and  4:  COMhANY  OPERATIONS 
Operators  at  these  stations  are  allocated  the 
responsibility  to  monitor  and  analyze  the  activites 
of  each  of  the  three  BLUEFOR  line  companies  and 
their  subordinate  platoons. 

STATION  5:  EXERCISE  OPERATIONS 
Operators  at  this  station  are  allocat’d  the 
responsiblity  to  monitor  and  control  the  training 
environment.  These  responsibilities  Include  di¬ 
recting  the  Field  Observer/Controllers  (FOC),  fire 
marker  teams  and  monitoring  the  status  of  the 
NTC-IS  Instrumentation  hardware  and  software. 
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STATION  6:  OPFOR,  BN  and  BOE  OPERATIONS 
Operators  at  this  station  are  assigned  the 
responsibility  to  direct  the  opposing  forces 
(OPFOR)  and  monitor  the  battalion  TF  and  brigade 
(BOE)  tactical  and  Intelligence  operations.  When 
nuclear,  biological  or  chemical  (NBC)  effects  are 
played,  the  NBC  operator  Is  accommodated  at  this 
station. 

STATION  7:  FIRE  SUPPORT  OPERATIONS 
Operators  at  this  station  are  assigned  the 
responsibility  to  monitor  and  direct  the  simulation 
of  Indirect  fire  operations  for  both  the  battalion 
TF  and  OPFOR. 

STATION  8:  SUPPORT  OPERATIONS 
Operators  at  this  station  are  assigned  the 
responsibility  to  monitor  and  analyze  all  battalion 
TF  combat  support  and  combat  service  support  opera¬ 
tions. 

Using  the  controller  station  capabilities 
previously  described,  the  EMC/TAF  operators  are 
able  to  perform  their  particular  function  In  the 
CIS  by  monitoring  the  appropriate  radio  nets, 
tagging  event  data  collected  by  the  field  Instru¬ 
mentation,  directing  and  prompting  the  operation 
and  information  input  of  the  Field  Observer/ 
Controllers,  dlectlng  the  OPFOR  operations,  di¬ 
recting  and  editing  the  activities  of  the  field 
video  teams  and  selecting  the  data  and  video 
displays  to  emphasize  and  demonstrate  specific 
training  objectives  for  AAR  sessions. 


SUMMARY  AND  SCHEDULE 

S  unwary 

The  National  Training  Center  (NTC)  then  has 
been  designed  to  provide  to  the  members  of  each 
battalion  a  total  experience  which  cannot  be  dupli¬ 
cated  at  their  homo  stations.  The  Instrumentation 
System  will  provide  the  means  by  which  the  training 
experience  can  be  controlled  and  the  means  by  which 
the  data  can  be  collected,  recalled  and  presented 
so  that  units  will  have  an  appreciation  of  their 
state  of  readiness  for  combat  and  can  develop  a 
training  program  to  eliminate  their  weaknesses. 

Not  only  will  the  NTC  provide  the  opportunity 
for  the  units  that  pass  through  the  14-day  period 
to  Improve  their  training  but  It  will  provide  an 
opportunity  for  the  Army  to  Identify  better  train¬ 
ing  methods.  It  will  provide  a  system  which  will 
allow  a  unit's  performance  to  be  judged  objectively 
In  a  realistic  combat  environment.  Home  station 
training  will  become  more  effective  as  a  result  of 
units  experiences  at  NTC  and  as  a  result  of  the 
periodic  reviews  of  their  NTC  performance  by  use  of 
their  Take-Home-Package  of  recorded  arterial.  The 
net  result  will  be  that  ove*  time  the  performance 
of  battalions  at  NTC  will  Improve  as  they  learn  to 
train  more  effectively  at  nomi  stations. 

Schedule 

Initial  operational  capability 
-  125  player  units  -  31  Jan  82 

Expanded  player  unit  capability  -  Oct  82 

Phase  II  implementation  -  Incrementally  from 

mid  to  late  1980's 
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ABSTRACT 


Pierside  combat  systems  training  Is  a  way  to  supplement  the  shore  based  training 
facilities  and  the  at-sea  exercises  which  are  so  necessary  to  keep  a  combat  team  at  a  high  level  of 
proficiency.  This  papsr  presentr  how  pierside  training  Is  being  accomplished  with  the  use  of  the 
Device  20B4,  Mobile  Combat  Systems  Trainer.  The  20B4  provides  the  capability  for  conducting 
Individual  operator  and  team  training  on  the  crew's  ownahip  through  stimulation  of  the  installed 
operational  equipment.  The  user  can  develop,  modify,  expand,  and  replay  training  exercises  quickly 
through  a  combination  of  real-time  and  off-line  software  routines.  The  system  presents  a  total 
electronic  warfare  environment  to  a  wide  range  of  sensors  with  a  cost  effective  adaptability  to  the 
sensors  and  threats.  The  mobility  of  the  20B4  enables  the  system  to  be  moved  wherever  the  ships  are 
located  to  achieve  improved  fleet  readlnes1;  through  realistic  training  and  checkout. 


INTRODUCTION 

The  classic  problem  in  warfare  ir  to  en..vre 
that  the  Military  personnel  end  equipment  ere 
organized,  trained,  and  ready  to  handle >a  vide 
variety  of  combat  situations.  This  la  sarticu- 
larly  true  for  naval  ships  where  complex  team 
operations,  actions.  Interactions ,  and  deci¬ 
sions  are  required.  The  precision  and  speed 
Involved  in  threat  detection  identification, 
engagement,  and  weapons  deployment  ere  critical 
to  the  survival  of  combat  ships.  The  coordi¬ 
nation  of  these  activities  in  a  battle  environ¬ 
ment  Is  complicated  and  crew  proficiency  la 
difficult  and  expensive  to  maintain  during 
peacetime.  To  achieve  a  high  level  of 
proficiency,  without  being  in  battle,  la  the 
goal  of  all  training  activities. 

Until  recently,  coordinated  combat  systems 
team  training  was  provided  to  varying  degrees 
at  land-based  training  facilities,  and  through 
at-sea  exercises  which  involved  the  use  of 
operational  aircraft  and  ships.  As  with  any 
training  situation,  there  were  limitations  to 
what  could  be  accomplished  under  these 
conditions.  The  problem  was  to  fill  the  gaps, 
bring  training  to  the  fleet,  and  accent  the 
coordination  of  the  combat  systems  team.  Among 
alternatives  considered  to  supplement  the  ahoie 
based  facilities  and  at -pea  exercises,  was  that 
of  a  pierside  combat  cyatems  trainer.  This 
device  would  create  a  realistic  threat 
environment,  directly  stimulate  ehJ^'vcrrd 
sensors,  and  Interact  with  the  operation  of 
these  sensors. 

The  result  would  he  a  trailing  system  that 
permitted  on-ship  training  of  combat  systems 
teams  using  their  shipboard  equipment  and 
training  scenario  coordination  provided  by 
knowledgeable  Fleet  training  personnel.  The 
Device  2044  Mobile  Csrabct  Syrtems  Trainer 
(MCST)  is  one  such  device. 


DESCRIPTION  OF  THE  20B4 


Functional  Description 

The  20B4  MCST  is  a  completely  mobile  unit, 
with  all  equipment  contained  in  an  air- 
conditioned  semitrailer  which  has  an  alr- 
cutJ  toned  suspension  system.  (ace  Figure  1, 
lecture  of  2004  Van.) 

The  software  programs  and  peripheral 
devices  compute  ill  target  and  vehicle  dynamics 
for  any  trailing  exercise.  Tnree-  dixsnsional 
’.andmaas  and  weather  formations  are  stored  on 
easily  accessed  magnetic  tape  and  disc 
cartridges.  Among  the  other  software  programs 
are  Loose  for  data  entry,  radar  parameter 
tables,  Electronic  Warfare  emitter  tablej,  and 
hardware  diagnostics  for  ease  of  maintenance. 
(See  Figure  2,  the  20B4  MCST  block  diagram.) 

Data  from  the  computet  is  passed  to  high 
speed  electronic  modules  designed  specifically 
for  the  20B4,  Each  module  performs  a  desig¬ 
nated  task,  such  as  target  attenuation.  This 
modularization  permits  easy  fault  Isolatlcr., 
and  correction  by  module  replacement.  The 
combination  of  meny  elements  of  target  charac¬ 
teristics  data  is  generated  In  the  proper 
format  for  application  to  varie-rr  types  or 
radar  systems  (such  as  puls®  Continuous  Wave, 
pulsed  dcppler,  chirped  pulse,  coded  pulse 
groups,  Frequency  Modulation,  Continuous  i'a”e. 
or  others).  Tnis  signal  is  provided  to  the 
appropriate  radar  channel  from  the  20B4  van. 
Fire  control  radar  type*  may  be  of  the  conical 
scan,  Conical  'Jest,  deceive  Only  scan,  simul¬ 
taneous  tnonopnlse  using  1,  2,  or  3  channel 

receiver  systems,  with  or  without  sidelobe 
cancellation  features. 

The  signals  ere  transmitted  to  the  ship 
through  1-inch  diameter  cables  (normally,  one 
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per  sensor),  which  terminate  in  sensor 
interface  units.  The  attache-case  sized 
interface  units  further  condition  and 
distribute  the  signals  through  a  harness  which 
is  configured  to  permit  quick,  temporary 
connection  to  the  sensor.  (See  Figure  3, 
Interface  Kit).  Stimulation  of  the  radars  is 
done  at  the  intermediate  Frequency  of  the  radar 
to  uye  as  much  cf  the  .adar  system  as 
reasonable,  and  to  achieve  realistic  interface 
for  effective  Electronic  Counter  Measures 
interactions,  A  side  benefit  cf  this  approach 
is  that  it  serves  as  a  systems  l.vel  device  to 
generally  assesa  the  operability  of  the  radar. 
EW  equipment  interface  is  provided  at  the  RF 
level  for  the  seme  reasons. 

in  its  present  configuration,  the  20B4 
simultaneously  stimulates  up  to  seven  radar 
sensors  and  various  Electronic  Warfare 
equipments.  The  radars  typically  include  a 
short  range  surface  search,  a  long  range  air 
search,  a  three-dimensional  air  search,  and 
four  missile  or  gun  fire  control  radars.  These 
radars  are  selected  from  the  list  of  23 
different  radars  currently  interfaced  by  the 
20B4.  The  Electronic  Warfare  equipment 
stimulated  include  countermeasures  sets, 
deception  repesters,  Direction  Finder  Sets, 
countermeasures  receivers,  and  threat 
identifiers.  The.  20B4  also  stimulates 
Identification  Friend  or  Foe  (IFF)  and  ownship 
motion  equipment. 

The  threat  environment  of  the  20B4 
presents  up  to  32  independent  targets 
simultaneously, ' such  as  ships,  aircraft  decoys, 
and  missiles.  In  any  given  scenario,  hundreds 
of  targets  can  be  presented  over  time.  These 
are  accompanied  by  appropriate  signatures, 
landmass,  weather,  sea  state.  Electronic 
Counter  Measures  and  chaff.  Each  threat  or 
environment  parameter  output  results  in 
presenting  the  shipboard  equipment  with  the 
static  or  dynamic  features  which  would  be 
associated  with  real  conditions ,  Three- 
dimensional  lancuasses  from  around  the  world 
are  available,  and  aircraft  features  include 
size,  aspect,  turn  rate,  speed,  and  accel¬ 
eration,  among  others.  Missiles  employed  in 
the  training  scenarios  feature  appropriate 
elements  such  as  Electtonlc  Warfare  emissions, 
range,  altitude,  and  velocity  characteristics. 
Janolng  conditions  can  be  automated  or  acti¬ 
vated  by  operators  during  exercises.  Targets 
that  fly  behind  the  landmass  mountains  are 
masked  as  in  real  life,  and  targets  which  are 
successfully  engaged  are  removed  from  the 
displays  when  they  are  destroyed. 

The  Electronic  Warfare  Simulator  (EWS)  of 
the  20B4  will  present  up  to  32  simultaneous 
threats  in  the  upper  bands  (A  through  J). 
Again,  in  any  scenario,  hundreds  of  Electronic 
Warfare  targets  can  be  presented  over  time. 
Each  Electronic  Warfare  threat  may  be  assigned 
a  unique  pulse  repetition  interval  and  pulse 
code,  and  there  are  no  restrictions  placed  on 
the  number  of  threats  in  any  particular  band. 
Each  emitter  may  have  any  of  10  basic  6csn 
patterns  (circular,  spiral ,  rente'-,  palmer, 


etc.)  generated  from  one  of  16  hardware  stored 
antenna  patterns.  Disc  storage  provides  for 
250  emitter  signatures  for  scenario  preparation 
and  Implementation. 

Sixteen  independent,  progiansable  jammers 
are  available  for  simultaneous  use.  Hundreds 
of  these  may  be  employed  during  the  course  of 
any  given  scenario.  They  provide  selectable 
features  such  as  carrier  and  amplitude 
modulation,  range  or  angle  gate  steal,  blink 
and  angle  deception.  Chaff  drops  are  also 
progransable  and  up  to  16  simultaneous  drops 
nay  be  made  at  any  time. 

The  20B4  stimulates  the  AIMS  MK  X  or  MK  XII 
IFF  systems  at  the  radio  frequency  level.  All 
modes  are  stimulated,  although  mode  4  is 
simulated  to  avoid  exercising  crypto-secure 
circuits . 

Ownship  motion  is  provided  for  any  one  of 
the  10  classes  of  ships  currently  in  the  20B4 
library.  The  overall  gaming  area  is  1,024  by 
1,024  nautical  miles  which  permits  a  great  deal 
of  flexibility  in  scenario  preparation. 

Pre-operatlng  Features 

All  information  used  to  prepare  an  exercise 
is  entered  via  the  Operator  Console  of  Device 
20B4.  (See  Figure  4,  Operator  Control 
Console.)  Two  cathode  ray  tube  displays  with 
keyboards  allow  the  instructor  to  create 
targets,  activate  jammers,  start  and  stop 
scenarios,  change  weather  conditions,  or  alter 
any  other  factors  that  would  affect  the  ship’s 
sensors.  A  Plan  Position  Indicator  is  provided 
to  3how  the  actual  display  being  presented  to 
the  ship's  radars.  Also,  an  A-acope  and  patch 
panel  Including  an  intermediate  frequency 
amplifier-detector  allow  the  instructor  to 
monitor  pertinent  signals  such  as  radar  sync, 
ranging,  video,  and  error  signals. 

The  importance  of  realistic  training 
scenarios  cannot  be  overstressed,  and  the  20B4 
permits  scenario  preparation  by  the  user,  based 
or.  his  training  objectives.  These  can  be 
prepared  off-line  through  use  of  the  scenario 
compiler,  and  instructions  are  entered  in  time, 
target  or  object,  function,  and  value  format. 
The  resultant  data  i«  stored  on  magnetic  tape 
(or  cards)  and  a  hard  copy  output  may  be 
obtained  from  a  line  printer  installed  in  the 
van.  Scenarios  may  also  be  generated  on-line 
through  Instructor  Manual  (Keyboard)  operations 
iiad  automatically  recorded.  Scenarios  may  be 
created  for  simple  or  complex  missions ,  to 
emphasize  some  specific  training  goal  or  for  a 
variety  of  other  ubcs. 

interface  with  the  ship  is  through  the 
Interface  Kits  previously  mentioned.  These  are 
electronically  keyed  and  automatically  alert 
the  20B4  Instructor  if  the  cable  inter¬ 
connections  are  incorrect. 

After  the  interface  is  completed  and 
verified,  the  20B4  and  ship  sensors  must  be 
correlated  through  alignment.  This  is  done  in 
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a  step-by-step  procedure  from  the  2084 
Instructor  position  and  includes  such  elenents 
as  azimuth  and  elevation  angle  offsets. 
Depending  on  the  type  of  ship  and  number  of 
sensors  to  interface ,  this  overall  Interface 
task  can  be  done  in  as  little  as  1  or  2  days. 
411  data  for  a  given  ship  is  recorded  on  disc, 
so  that  it  is  readily  recalled  for  that  ship 
when  training  is  conducted  at  a  later  time. 

The  final  step  is  to  initialize  the 
training  program.  This  is  done  by  selecting 
the  scenario,  landmass  map,  weather  map,  and 
weather  height .  This  may  be  done  from  t'ae 
Operator's  position.  The  Operator  then  selects 
the  appropriate  mode  of  operation,  NORMAL  or 
RECOl’D.  The  RECORD  mode  is  used  to  generate  a 
scenario  on-line . 

Operation 

Following  interface  and  alignment,  the  2084 
may  be  used  to  run  as  many  scenarios  as 
training  time  permits. 

Operation  in  the  NORMAL  mode  consists  of 
starting  a  prerecorded  scenario  and  monitoring 
the  sensor  operator  action  and  responses.  For 
greater  realism,  there  is  a  second  Operator 
position  from  which  the  Combat  Air  Fatrol  (CAP) 
simulated  aircraft  is  controlled.  Due  to  the 
dynamics  of  the  CAP  interactions  with  the 
ships'  Air  Intercept  Control  personnel,  both 
target  control  and  communications  are  handled 
by  this  Operator.  Both  positions  for  the  20B4 
and  CAP  instructors  are  identical,  and  each  has 
access  to  all  displays  as  well  as  the 
capability  to  manually  Intervene  to  modify  the 
scenario,  if  desired.  The  use  of  the  prepared 
scenarios  thus  minimizes  the  operating 
functions  to  be  performed,  but  the  system 
offers  the  potential  to  modify  the  scenario 
during  training  as  desired.  Further,  the  use 
of  standardized,  repeatable  scenarios  can 
provide  relative  personnel  training  assessments 
or  evaluation  of  combat  systems  capabilities  of 
various  classes  of  ships. 

Data  Collection  and  Evaluation 

During  the  training  exercises  (and  keyed  to 
the  specific  scenario),  there  are  a  number  of 
Important  events  that  are  recorded.  These  are 
automatically  obtained  from  flagged  scenario 
data,  sensing  points  in  the  interface  or  from 
Inputs  derived  from  hand-held  keysets 
controlled  by  instructor/observers  positioned 
at  critical  locations  aboard  ships.  The  data 
recorded  includes  administrative  information 
such  as  the  date,  ship,  time,  as  well  as  other 
inputs.  Electronic  Warfare  data  which  can  be 
recorded  includes  emitters  presented, 
identified  correctly,  engaged  correctly  and  the 
time  to  detect  high  priority  threats.  Radar 
target  data  which  can  be  recorded  includes 
targets  presented,  engaged,  and  killed  by 
threat  class,  average  detection  time,  average 
target  assignment  time,  as  well  as  Search  and 
Track  radar  summary  information.  This  data  may 


be  used  to  Identify  specific  problem  areas, 
debrief  ships  crews  and  officers,  or  assess 
overall  effectiveness  for  readiness  evaluation. 
Due  to  the  complexities  of  evaluation,  the  data 
collection  use  is  strictly  up  to  the  user. 

20B4  MCST  Expandability 

Although  the  20B4  currently  supports  a 
defined  set  of  sensors,  additional  units  can  be 
stimulated  with  very  little  modification.  The 
20B4  provides  stimulation  for  many  types  of 
radar ,  as  well  as  Electronic  Warfare  signatures 
within  the  radio  frequency  spectrum  from  0.2  to 
18  GHz.  To  stimulate  a  new  sensor,  the  general 
technique  is  to  develop  a  new  interface  or 
modify  an  existing  one. 

The  capability  to  provide  anti-submarine 
warfare  training  has  been  taken  into  account  in 
the  basic  20B4  design.  Should  there  be  a 
requirement  for  this  training  mission  to  be 
assigned  to  the  20B4,  there  is  enough  physical 
space  and  system  computing  time  to  accommodate 
it. 

Conclusion: 

Pierside  combat  Bystems  training  is  an 
important  element  in  the  present  training 
concept  for  Navy  shipboard  personnel.  The 
strong  point  of  pierside  training  is  that  it 
brings  the  environmental  control  of  the 
classroom  to  the  actual  shipboard  equipment  and 
combat  teams.  The  20B4  has  been  used 
successfully  as  a  cost  effective  means  of 
maintaining  combat  crew  proficiency  for  a 
number  of  ship  types  including  aircraft 
carriers,  cruisers .destroyers ,  and  frigates 
during  the  past  two  years  at  a  number  of  east 
coast  Naval  bases. 

F.xperience  to  date  Indicates  that  ship's 
crews  welcome  the  training  experience  because 
they  are  Integrated  as  combat  crews  into 
composite  training  exercises  in  which  these 
crews  are  given  an  opportunity  to  Interrelate 
and  cooperate  under  psuedo-combat  conditions  to 
successfully  complete  a  training  mission. 

Additional  20B4  MCST  units  are  currently 
being  fabricated  and  deployed  at  selected 
locations  throughout  the  world  for  more 
widespread  economical  proficiency  training. 

The  Device  20B4  is  not  limited  to  pierside 
combat  systems  training.  It  is  a  general 
purpose  radar  and  EW  stimulator/  trainer  and  is 
a  natural  extension  of  the  AN/MPQ-T1  Nike 
Hercules  Radar  Simulator  previously  designed  by 
AAI.  (The  AN/MPQ-T1  trainer  is  a  trailer 
mounted  system  \’hich  can  be  readily  moved  from 
missile  site  tc  missile  site  to  provide 
training  to  the  weapon  system  operators.)  With 
the  addition  of  new  interface  kits,  the  Device 
20B4  could  provide  individual  or  team  training 
to  almost  any  radar  or  Electronic  Warfare 
systems.  This  could  include  air,  land,  or  sea 
based  electronic  sensing  platforms. 
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ABSTRACT 


A  fundamental  problem  In  the  training  of  artillery  crews  Is  the  cost  of  ammunition, 
fuel  and  range  facilities  used  In  live  fire  training.  Current  dry  fire  training  la  slow  and 
monotonous  because  error  sources  can  only  be  detected  by  stopping  the  exercise  after  each  weapon 
laying  and  visually  Inspecting  the  various  sighting  and  ammunition  preparation  functions. 
Clearly,  there  is  a  need  for  a  training  device  which  provides  for  a  real-time  assessment  of 
weapon  laying  and  ammo  preparation  errors  and  which  is  compatible  with  current  live  and  dry  fire 
training  techniques.  This  paper  describes  the  AAI  Firing  Battery  Trainer,  a  device  capable  of 
training  each  crew  member  of  a  firing  battery  in  the  weapon-laying  and  ammo  preparation 
operations 


INTRODUCTION 


The  Firing  Battery  Trainer  ( FBT) ,  developed 
by  MI  under  the  direction  of  the  Human  Engineer¬ 
ing  Laboratory,  provides  the  equipment  necessary 
for  Instruction  and  practice  in  development  of  the 
skills  necessary  to  operate  the  howitzer  effec¬ 
tively  in  either  a  dry  or  live  fire  environment. 
The  FBT  is  a  system  that  Is  installed  on  the 
crew's  assigned  M109's,  thereby  developing  the 
realism  that  cannot  be  achieved  otherwise. 

The  FBT  is  configured  to  permit  the  training 
of  an  entire  battery  simultaneously.  A  single 
instructor  or  battery  commander  can  monitor  the 
speed  and  accuracy  of  each  gun  crew  from  a  single 
instructor's  console.  Any  number  of  weapons  from 
one  to  eight  can  be  connected  to  the  system.  A 
detailed  display  of  any  one  gun  crew's  error  data 
is  available  to  the  instructor  on  his  console,  and 
a  complete  printout  is  available  at  the  completion 
of  the  exercise. 


BACKGROUND 


There  are  two  groups  of  laying  errors  which 
affect  laying  accuracy.  They  are:  1)  deflection 
errors  and  2)  quadrant  errors.  Deflection  errors 
are  errors  in  the  azimuth  of  the  gun  tube  commit¬ 
ted  by  the  gunner.  Quadrant  errors  are  errors  iu 
the  elevation  of  the  gun  tube  committed  by  the. 
assistant  gunner. 

In  laying  the  weapon  in  azimuth,  there  are 
four  operations  to  be  performed.  Two  of  the  oper¬ 
ations  have  a  direct  effect  on  gun  tube  placement. 
They  are:  1)  entering  the  commanded  deflection 
into  the  sight,  and  2)  rotating  the  turret  to  the 
entered  deflection.  An  error  of  one  mil  in  either 
of  these  operations  results  in  a  one  mil  error  in 
gun  tube  azimuth.  The  remaining  two  operations, 
3)  levelling  and  4)  cross-levelling,  have  a  second 
order  effect  on  gun  tube  azimuth. 

In  laying  the  weapon  in  elevation,  there  are 
three  operations  to  be  performed.  Two  of  the 
operations  have  a  direct  effect  on  gun  tube  place¬ 
ment.  They  are:  1)  entering  the  commanded 


quadrant  into  the  sight,  and  2)  elevating  the  gun 
tube  to  the  entered  quadrant.  An  error  of  one  mil 
in  either  operation  results  in  a  one  mil  error  in 
gun  tube  elevation.  The  remaining  operation,  3) 
X-levelling,  has  a  second  order  effect  on  gun  tube 
elevation. 

In  preparing  the  ammo ,  there  are  four  opera¬ 
tions  to  be  performed.  They  are:  1)  projectile 
type  selection,  2)  fuze  type  selection,  3)  fuze 
time  entry,  and  4)  charge  number  and  charge  color 
selection.  With  the  possible  exception  of  fuze 
time,  ammo  preparation  errors  will  reBult  in  a 
significant  change  in  the  kill-capability  of  a 
given  artillery  round. 


DESCRIPTION  OF  THE  FBT 


The  FBT  system  has  the  capability  of  being 
used  for  either  live  fire  or  dry  fire.  All 
weapon-mounted  equipment  is  capable  of  with¬ 
standing  both  the  shock  associated  with  howitzer 
fire  and  vibration  associated  with  howitzer 
travel.  Other  components  are  designed  to  with¬ 
stand  abusive  treatment  resulting  from  the  use  of 
the  equipment  in  the  field.  For  dry  fire  train¬ 
ing,  the  howitzer  and  trainer  can  be  used  in  a 
"parking  lot”  environment.  See  Figure  1. 

Weapon  Mounted  Components 

The  FBT  is  basically  a  measuring  device  with 
feed-back.  Its  function  is  accomplished  by 
instrumenting  the  Ml  17  pantel  and  M15  quadrant  and 
providing  a  hand  operated  unit  to  indicate  ammo 
preparation  errors,  all  connected  to  an  instruc¬ 
tor's  console  for  control  of  the  training  exer¬ 
cises.  The  pantel  is  modified  by  adding  sensors 
to  detect  the  azimuth  entered  value,  sight  pat¬ 
tern,  level  and  cross-level  and  by  addins  hi/ low 
lights  to  provide  feed-back  if  an  error  exists. 
The  quadrant  is  likewise  modified  to  detect 
entered  value,  level  and  cross-level  and  by  adding 
hi/low  lights.  Ammunition  preparation  activity  is 
detected  by  observing  the  crew  and  entering 
error/no  error  on  the  hand  held  monitoring  unit. 
Possible  errors  are  projectile  type,  fuze  type, 
fuze  time  and  charge.  Each  of  the  above  items  is 
connected  to  a  weapon-mounted  interface  unit  for 
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signal  processing  and  transmission  to  the  instruc¬ 
tor's  console.  In  addition,  the  Interface  unit 
provides  the  Chief  of  Section  with  a  summary  of 
the  seven  laying  errors. 


Instructor's  Equipment 


The  Instructor's  console  la  composed  of  two 
sections,  the  error  display  unit  nnd  the  control 
unit.  The  display  Is  located  In  the  lid  of  the 
console  and  the  control  unit  in  the  base.  The 
display  provides  a  complete  summary  of  all  laying 
and  ammo  errors,  along  with  the  comanded  values, 
for  one  weapon.  The  error  data  Is  displayed  real 
time,  l.e.,  as  the  gunner  and  assistant  gunner 
perform  their  operations,  tne  error  display  will 
show  the  movement  of  the  sights,  turret,  and  gun 
tube.  All  laying  errors  are  displayed  with  en 
accuracy  of  0.1  mils,  except  the  sight  pattern 
error  which  has  an  accuracy  of  0.2  alls. 


The  control  section  of  the  Instructor' t  con¬ 
sole  permits  the  Instructor  to  control  1)  the 
training  mission,  2)  the  input  of  commend  data,  3) 
the  display  of  information  on  the  error  display, 
and  4)  the  display  of  the  on-carrlage  hi/low  error 
lights.  Prompting  provides  the  instructor  with 
all  options  available  to  him  for  each  data  Input. 

The  printer  unit  serves  two  functions. 
First,  lc  contains  the  power  conditioning  system 
for  the  entire  weapon  trainer  system.  The  system 
operates  from  ll5vae  60  Hz  10  amps  (standard  house 
power  in  the  USA) .  The  second  function  la  to 
house  the  printer  unit.  The  printer  Is  used  to 
provide  Instructor  and  students  with  a  record  of 
their  performance. 


Technical  Description 


The  FBT  contains  sensors  or  devices  to  mea¬ 
sure  a  total  of  eleven  error  sources  from  each 
howitzer.  They  are: 


(A)  Deflection  Entered  Value 

(B)  Deflection  Sight  Pattern 

(C)  Deflection  Level 

(D)  Deflection  XLevel 

(E)  Quadrant  Entered  Value 

(F)  Quadrant  Level 

(G)  Quadrant  XLevel 

(H)  Projectile  Type  Selection 

(I)  Fuze  Type  Selection 

(J)  Fuze  Time  Setting 

(K)  Charge  Preparation 

The  measurement  of  these  error  sources  is 
accomplished  by  four  different  measurement  tech¬ 
niques.  They  are: 

(A)  Level/XLevel  Measurement  (Quadrant  and 
Deflection) 

(B)  Entered  Value  Measurement  (Quadrant  and 
Deflection) 

(C)  Sight  Pattern  Measurement 

(D)  Ammo  Preparation  Errors  (Projectile 
Type,  Fuze  Type,  Fuze  Time,  and  Charge) 

The  four  level/Xlevel  error  sources  lisbed 
above  are  measured  and  processed  In  a  similar 
fashion,  utilizing  the  same  type  of  sensor  and 
signal  processing  techniques.  The  sensing 
devices,  or  inclinometers,  are  gravity  referenced 
devices  which  provide  a  bl-polar  analog  DC  voltage 


output  which  la  piooortlonal  to  the  angle  of  tilt. 
The  aenaora  uaad  on  kha  pantel  and  quadrant  ara 
essentially  tha  aame,  differing  only  In  their 
physical  size,  weight,  and  output  range.  Once 
digitized,  the  aensltlvity  of  both  sensors  la  0.) 
mil  per  LSB. 

The  two  entered  value  aenaora  are  optical 
shaft  angle  measurement  devices  of  the  incremental 
type,  Tha  aenaora  used  on  the  pantel  and  quadrant 
are  essentially  the  same,  but  because  the  two 
sight*  have  different  Internal  gearing,  the  sen- 
aore  must  have  correspondingly  different  resolu¬ 
tions  in  order  to  provide  the  aame  0.1  mil  per  bit 
error  resolution. 

The  sight  pattern  sensing  ayatem  consists  of 
an  optical  Image  processing  ayatem  followed  by  an 
IR  detector.  The  signal  from  the  detector,  when 
digltzed  end  processed  by  the  computer  ayatem, 
provides  a  0.2  mil  accuracy  determination  of  tur¬ 
ret  asimuth. 

The  computer  system,  which  consists  of  the 
computer  and  I/O  controller  is  the  control  element 
of  the  FBT.  All  data  communications  between  sen¬ 
sors,  displays,  and  keyboards  are  controlled  by 
the  computer  system.  Memory  mapped  I/O  techniques 
are  used  for  ell  computer  system  Interfaces,  ex¬ 
cept  the  printer  Interface  which  is  via  an  RS-232 
link.  All  software  was  written  in  assembly  lan¬ 
guage  In  order  to  minimize  memory  requirements  and 
optimize  operating  efficiency.  The  FBT  software 
is  divided  Into  two  subsystems:  Initialization 
software  and  training  mission  software.  The  ini¬ 
tialization  software  Is  written  as  an  off-line 
system,  while  the  training  mission  software  Is  an 
Interrupt  driven  real-time  system. 


OPERATION 


Prior  to  the  start  of  a  training  exercise , 
the  FBT  must  be  set  up  and  initialized.  Either 
the  one-step  or  the  two-step  Initialization  pro¬ 
cedure  may  be  selected.  Once  Initialization  has 
been  completed,  the  training  exercise  can  begin. 

Initialization 

FBT  System  initialization  falls  into  two 
categories:  first,  FBT  system  Initialization, 
and,  second,  sensor  initialization.  System 
Initialization  consists  of  entering  the  sight 
Initialization  values,  day,  and  time  of  day. 

The  FBT  sensors  can  be  Initialized  by  fol¬ 
lowing  either  of  two  procedures  depending  on  the 
placement  of  the  collimator.  See  Figure  2.  Each 
procedure  has  its  own  merits  and  limitations. 
Placement  of  the  collimator  "on-line"  Is  more 
difficult,  but  enables  FBT  initialization  to  be 
accomplished  with  a  simple  one  step  procedure. 
Placement  of  the  collimator  "off-line”  is  easier 
but  requires  a  more  time-consuming  two-step  FBT 
Initialization  procedure. 

The  one-step  initialization  procedure  can  be 
used  if  the  collimator  is  emplaced  at  33  feet  and 
on  the  extension  of  the  line  connecting  the  pantel 
and  the  turret  center-of-rotation.  While  on-line 
emplacement  might  be  difficult  or  Impossible,  due 
to  the  local  terrain  variations,  it  Is  the  only 

554 


collimator  location  which  will  permit  360  degrees 
turret  laying. 

After  orienting  the  battery  with  the  M2  aim¬ 
ing  circle  and  emplacing  the  Ml  collimator,  the 
pantel  and  the  quadrant  is  set  at  3200  and  0300. 
The  Chief  of  Section  must  depress  his  "COS  Ready" 
switch  to  complete  FBT  sensor  initialization. 

The  two  step  Initialization  procedure  was 
developed  to  permit  greater  flexibility  in  the 
placement  of  the  collimator.  With  this  procedure, 
the  collimator  can  be  placed  at  any  angle  between 
2020  and  2420  mils  at  33  feet.  Turret  rotation  is 
limited  when  the  collimator  is  placed  off  line, 
but  the  FBT  sight  pattern  detector  system  has  the 
same  field  of  view  as  the  pantel;  i.e,,  the  FBT 
will  be  able  to  measure  the  sight  pattern  at  any 
turret  angle  at  which  the  gunner  can  see  it. 

The  two  step  procedure  requires  that  the 
weapon  be  laid  to  two  deflection  commands,  3200 
and  2100  mils  for  the  M109.  The  first  lay,  3200/ 
300,  is  automatically  performed  when  the  gun  is 
oriented  with  the  aiming  circle.  The  second  lay, 
2100/300,  is  equivalent  to  a  lay  command  at  the 
indicated  values.  All  critical  parameters 
(levels,  Xlevels  and  entered  values)  are  monitored 
to  assure  that  they  are  within  the  required  toler¬ 
ance.  If  any  one  parameter  is  out-of-tolerance 
for  either  step,  then  the  FBT  will  inform  the 
instructor.  The  gun  crew  must  then  correct  this 
parameter  before  that  initialization  step  can  be 
completed.  At  the  completion  of  each  step,  the 
COS  must  depress  his  "COS  Ready"  switch  in  order 
to  proceed . 

Operation 

Once  initialization  has  been  completed,  the 
system  is  ready  to  begin  a  training  mission.  The 
pace  and  format  of  a  training  mission  is  under  the 
complete  control  of  the  instructor.  All  firing 
commands  are  entered  by  the  instructor,  and  he 
selects  the  format  (fixed  target/moving  target,  on 
carriage  prompting  lights  on/of c,  rapid  fire/slow 
fire,  etc.)  with  the  Mission  Keyboard. 

Due  to  memory  limitations  in  the  FBT  compu¬ 
ter,  a  maximum  of  ten  lay  commands  may  be  Issued 
for  any  one  mission.  This  mission  may  be  set  up 
as  either  a  fixed  target  or  moving  target  mission. 
A  fixed  target  mission  is  a  "fire  when  ready" 
format.  Just  prior  to  firing  the  round,  the  COS 
must  depress  the  "COS  Ready"  switch  on  his  COS 
interface  unit  in  order  to  signal  the  FBT  (and  the 
instructor)  that  the  weapon  is  laid.  In  order  to 
implement  a  fixed  target  mission,  the  instructor 
L)  depresses  the  "F.nter  New  Command"  button  of  the 
Mission  Keyboard  2)  enters  the  commands  via  the 
Data  Keyboard,  3)  depress  the  "Issue  Command" 
button  of  the  Mission  Keyboard,  and  simultane¬ 
ously,  4)  issues  the  commands  to  the  gun  crews. 
At  this  point,  the  FBT  monitors  the  laying  opera¬ 
tion  and  awaits  the  COS  ready  signal  from  the  gun 
crew.  This  signal  indicates  that  the  round  has 
been  fired  ("shot  out")  for  fixed  target  missions. 
When  the  signal  Is  received  (or  when  the  59  second 
time  out  has  been  reached),  the  Instructor  can 
either  enter  a  new  set  of  commands  by  depressing 
the  "Enter  New  Command"  switch  and  then  repeating 
the  above  procedure,  or  end  the  mission  by  depres¬ 
sing  the  "EOM"  switch.  The  instructor  may  issue 
as  many  as  ten  commands  during  any  one  mission. 


For  moving  target  missions,  there  is  a  "Fire 
On  Command"  order  given.  This  means  that  the 
weapon  is  not  fired  until  a  separate  fife  order  is 
given  by  the  instructor.  To  implement  a  moving 
target  mission  on  the  FBT,  the  procedure  is  iden¬ 
tical  to  the  fixed  target  procedure  discussed 
above,  with  one  exception.  Upon  receipt  of  the 
COS  ready  signal,  the  instructor  may  either  fire 
the  round  or  issue  another  set  of  lay  commands. 
To  fire  the  round,  the  Instructor  depresses  the 
"Fire"  button  on  the  Mission  Keyboard  and,  simul¬ 
taneously,  gives  the  fire  order  to  the  gun  crew. 
To  issue  another  set  of  lay  commands,  the  instruc¬ 
tor  depresses  the  "Enter  New  Conmand"  button  on 
the  Mission  Keyboard  and  then  follows  the  same 
procedure  followed  for  fixed  target  missions 
above.  The  decision  to  fire  the  round  or  Issue 
another  set  of  lay  conmands  is  based  on  lay  time 
and  is  under  the  control  of  and  at  the  discretion 
of  the  instructor. 

When  starting  a  new  mission,  the  instructor 
has  the  option  of  enabling  the  on-carriage  indi¬ 
cators  and  setting  the  threshold  at  which  they 
come  on.  These  Indicators  provide  the  gunners 
with  immediate  feedback  and  prompting  of  their 
errors.  To  enable  these  indicators,  depress  the 
"On-Carriage  Indicators"  switch  of  the  Mission 
Keyboard.  When  the  built-in  light  is  illuminated, 
the  function  is  enabled. 

The  “pace"  of  the  training  mission  is  set  by 
the  rate  at  which  the  instructor  issues  commands. 
The  print-out  contains  the  "Start  Mission"  and 
"End  of  Mission"  times,  which  provides  an  indica¬ 
tion  of  the  pace  of  the  mission. 

SUMMARY 


It  is  possible  to  incorporate  the  FBT  into  a 
number  of  training  scenarios  which  permit  more 
interesting  and  cost-effective  training.  The  FBT 
provides  tangible,  documented  goals  which  each 
gunner  can  strive  to  achieve.  The  ability  to 
train  with  the  FBT  in  both  the  live  fire  and  dry 
fire  environment  allows  the  gun  crews  to  be  ex¬ 
posed  to  both  the  realism  of  live  fire  while  main¬ 
taining  proficiency  with  cost-effective  dry  fire 
scenarios.  In  addition,  the  FBT  permits  the 
instructor  to  develop  various  "games"  to  maintain 
the  crew's  interest.  Games  such  as  "beat  the 
clock"  and  "competition  among  crews"  provide  both 
valuable  training  and  increased  interest.  Future 
potential  for  the  FBT  Includes  integration  of  the 
trainer  with  larger  system  trainers.  Such  a 
training  system  could  pinpoint  poorly  trained 
troops  in  realistic  battlefield  scenarios. 

The  FBT,  having  completed  its  development 
cycle,  is  now  awaiting  its  first  field  evaluation. 
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ABSTRACT 

With  the  advent  of  real-time  interactive  combat  simulation  on  the  Advanced 
Simulator  for  Pilot  Training  (ASPT),  a  requirement  for  determining  weapon  effectiveness 
against  moving  threats  was  established.  Traditional  methods  either  required  an 
excessive  amount  of  computer  memory  or  were  restricted  to  low  fidelity  approximations. 
An  innovative  approach  to  this  problem  was  developed  for  ASPT.  An  iterative  approach 
utilizing  both  an  aerodynamic  model,  based  upon  the  weapon  ballistics,  and  the  threat 
position  time  nistory  serves  as  a  framework  for  this  method.  An  exact  determination  of 
weapon  Impact  or  miss  can  be  made  through  the  use  of  kinetics  and  calculus.  This 
method  allows  real-time  interactive  scenarios  that  Include  evasive  maneuvers  and  ECM 
tactics,  yet  requires  very  little  computer  memory  and  execution  time.  This  capability 
is  essential  for  effective  and  realistic  combat  simulation. 


INTRODUCTION 

Background 

Flight  simulators  are  being  used  more  and  more 
to  train  pilots  in  weapons  delivery  both  on  the 
conventional  range  and  in  the  hostile  environment. 

A  means  of  scoring  the  weapon  projectile  is 
essential  in  order  to  evaluate  pilot  performance. 

If  the  target  is  fixed  on  the  terrain,  scoring  is 
relatively  simple.  When  the  banb  or  bullet  is 
released,  the  flight  path  and  point  of  impact  with 
the  terrain  can  be  determined,  and  a  miss  distance 
can  be  obtained  that  Is  Independent  of  time. 
However,  when  the  target  Is  moving,  real-time 
scoring  becomes  much  more  difficult.  A  time 
history  of  the  target  and  the  weapon  projectile  is 
needed  so  that  the  distance  between  projectile  and 
target  may  be  evaluated  as  a  function  of  time. 
Depending  upon  the  fidelity  of  the  model  used  to 
create  this  time  history,  massive  amounts  of 
computer  memory  and  execution  time  may  be  required. 

The  most  common  method  currently  in  use 
involves  updating  the  weapon  projectile  position 
on  each  iteration.  This  method  is  very  accurate 
but  requires  a  large  amount  of  computer  memory  and 
execution  time,  especially  when  the  weapon  is  a 
stream  of  cannon  shells,  because  each  round  must  be 
updated  on  every  iteration.  Even  when  the  weapon 
and  target  position  are  known  for  each  iteration, 
interpolation  must  be  used  to  find  the  position 
between  iterations.  This  interpolation  and  the 
techniques  used  for  scoring  the  time  histories  of 
the  weapon  projectile  and  target  require  still 
more  computer  memory  and  execution  time. 

Statement  of  Problem 

In  a  real-time,  high  fidelity  threat  environ¬ 
ment,  computer  memory  and  execution  time  are  at  a 
premium.  Vet,  in  this  same  environment,  an 
accurate  weapon  scoring  algorithm  is  essential. 

The  ability  to  determine  weapon  effectiveness 
against  stationary  and  mobile  targets  is  a 
requirement  if  aircrew  training  or  survivability 
are  to  be  evaluated.  Scoring  against  stationary 
targets  Is  a  straight  forward  evaluation  of  weapon 
ballistics.  The  problem  of  weapon  scoring  against 
mobile  targets  is  one  of  finding  an  accurate  method 
that  requires  very  little  computer  memory  or 


execution  time.  An  innovative  approach  to  the 
problem  of  mobile  target  scoring  has  been 
developed  for  use  on  the  Advanced  Simulator  for 
Pilot  Training  (ASPT). 

METHOD 

Scoring  constitutes  a  measurement  of  the 
distance  from  the  target  to  the  weapon 
projectile.  The  ASPT  weapon,  whether  it  is  a 
bullet  or  a  missile,  is  treated  as  a  point  in 
space  located  at  the  front  edge  of  the  projectile, 
and  the  target  is  treated  as  a  sphere  whose 
diameter  is  approximately  the  wingspan  of  the 
target  (or  other  appropriate  cross-sectional 
dimension  depending  upon  the  object)  with  the 
center  of  the  sphere  located  at  the  target  center 
Of  gravity.  Consequently,  the  issue  is  whether 
the  weapon  (point)  penetrates  the  target  (sphere). 
At  the  present  time,  ASPT  does  not  require  fidelity 
to  the  degree  of  determining  exactly  where  the 
weapon  strikes  the  target.  However,  this  could  be 
found  by  applying  the  target  Euler  angles  (roll, 
pitch,  yaw)  to  the  target  sphere  and  redefining 
the  sphere  as  a  three-dimensional  object  that  has 
the  same  size  and  shape  as  the  target. 

Scoring  Algorithm 

The  method  for  ASPT  mobile  target  scoring 
determines  whether  the  target  sphere  is  penetrated 
by  the  weapon.  The  calculations  involved  determine 
the  exact  time  the  weapon  projectile  is  closest  to 
the  target  and  the  distance  between  them  at  that 
time.  These  calculations  are  the  same  regardless 
of  the  weapon  that  is  used. 

The  distance  between  the  weapon  and  the  target 
at  any  given  time  can  be  found  by  subtracting  the 
target  posit’"'  from  the  bullet  position.  These 
positions  ano  e  difference  are  treated  as  vectors 
in  three-dimensional  space. 

Equation  #1:  "5  *  Pweap  -  ptarg 

D  is  distance 
^weap  Is  weapon  position 
"Ptarg  1s  target  positon 
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The  weapon  and  target  position  at  any  time  can 
be  found  using  Equation  #2.  This  equation  assumes 
a  constant  velocity,  an  assumption  that  will  be 
discussed  in  more  detail  later. 

Equation  #2:  V  *  P0  ♦  VT 

7  is  position  at  time  T 

F0  is  position  at  time  T  *  0 

V  is  velocity 
T  is  time 

Substituting  Equation  #1  into  Equation  #2  and 
expressing  in  terms  of  the  coordinates  of 
three-dimensional  space  yields: 

Equation  #3a: 


Combining  Equation  15  and  Equation  #4 
gives: 

Equction  #6:  q^«(P  ^  +  p 

'  ox  oy  oz  ' 

*  <«>  <po.  \  ♦  v »,  ♦  pM  V  . 

*  (,«2  *  *,2  *  vz2)  (,2) 

This  is  a  second  order  equation  in  time 
whose  first  derivative,  when  set  equal  to  zero, 
will  give  a-point  of  minimum  or  maximum.  The 
equation  for  the  first  derivative  Is  as  follows: 

Equation  #7: 


2(P  JtP  V+P  V  )  +  2  (V2*  V2+  V2)  T  •  0 

v  ox  x  oy  y  oz  i'  '  x  y  V 


Ox  “  (pox  +  Vx^lweap  ■  (Pox  +  vx^)targ 
Equation  #3b: 

Oy  *  (Poy  +  VyHweap  -  (P0y  +  VyT)targ 
Equation  #3c: 


°z  =  (poz  +  ^zT)weap  -  (poz  +  vzT)targ 

The  total  distance  can  be  found  by  applying 
the  theorem  that  states  the  magnitude  of  a  vector 
squared  is  equal  to  the  sum  of  the  components 
squared. 

Equation  #4:  D2  *  D2  +  D2  +  O2 

x  y  z 


Before  substituting  Equation  #3  into 
Equation  #4,  it  would  be  helpful  to  rewrite 
Equation  #3  as  follows: 


D  =  (P 
x  '  ox. 


-  P 


weap 


ox 


targ 


)  *  (Vv 


-  v„ 


weap 


targ 


)  T 


D  =  (P 
y  oy, 


-  P 


weap 


oy 


targ 


)  +  (V, 


'weap 


Vy  -  )  T 

Jtarg 


Whether  the  point  is  a  minimum  or  maximum 
can  be  determined  by  looking  at  the  sign  of  the 
coefficient  of  t.«e  T  term.  This  is  always  a 
positive  number  so  the  point  will  be  a  minimixn. 

By  solving  for  T,  the  time  when  the  distance  will 
be  smallest  can  be  found. 

Equation  #8: 

T  .  -  <Pox  »,  *  P.y  »,  *  P»z  V 

(»/  *  V,p  *  »/) 

Taking  the  time  found  in  Equation  #8  and 
substituting  it  into  Equation  16  gives  the  miss 
distance,  the  closest  the  weapon  ever  gets  to  the 
target.  These  equations  are  based  on  the 
assumption  that  the  target  and  weapon  projectile 
are  under  no  acceleration.  This  assumption  is 
obviously  not  valid  for  all  time;  however,  for  a 
very  short  time  period,  it  can  be  considered 
valid.  In  the  next  section,  this  assumption  as  it 
applies  to  the  target  will  be  investigated. 
Following  that  discussion,  the  gun  and  missile 
applications  will  be  reviewed  and  particular 
characteristics  of  these  systems  addressed. 


z  *  Vz  )J 
weap  targ 


D  =  (P  -  P  )  +  (V 

z  '  oz  oz.  ' 

weap  targ 


Now  let: 

P  =  (P  -  P 

ox  '  ox_.n  ox. 

weap 

P  =  (P  -  P 

oy  %eap  oy 

P  *  (P  -  P 

oz  v  oz_,„  oz 

weap 

With  these  substitutions.  Equation  #3 

becomes: 


), 

'targ 

V  =  (V 

v  'X 

weap 

■  V, 

1  *  * 

vu  =  (v« 

-  V 

targ 

y  yweap 

narg 

,  ). 

targ 

Vz  *  (Vz 

weap 

-  vz 

ztarg 

Equation  #5a: 
Equation  #5b: 
Equation  #5c: 


D  =  P  +  V  T 
x  ox  x 


°y  "  P0y  *  V 

°Z  +  Poz  +  V 


Target 

A  no- acceleration  assumption  is  valid  when  a 
sufficiently  short  time  period  is  used.  For  a 
time  period  to  be  "sufficiently  short,"  it  must  be 
short  enou^i  so  that  any  normal  acceleration  the 
target  would  undergo  could  not  have  a  significant 
effect  on  the  location  of  the  target  at  the  end  of 
that  time  period.  Table  1,  found  In  the  Appendix, 
shows  the  effects  of  airspeed  and  G- loadings 
(acceleration)  on  target  displacement  for  varying 
time  periods. 

Taking  one  line  from  Table  1  as  an  example, 
enter  the  table  with  500  knots  true  airspeed 
(KTAS).  At  this  airspeed,  we  will  assume  the 
aircraft  is  limited  to  approximately  7  Gs  by 
either  structural  limits  or  aerodynamic  stall. 
Using  a  value  of  7  Gs  and  choosing  a  time  period 
of  one-tenth  of  a  second,  the  lateral  displacement 
is  1.126  feet.  That  means  the  difference  between 
where  the  target  would  be  located  if  it  were 
accelerated  for  one-tenth  of  a  second,  and  where 
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It  would  be  If  It  were  not  accelerated  for  one- 
tenth  of  a  second  1$  1.126  feet.  (See  figure  1.) 


Time  *  o 


— y  i.i26  ft 

Time  ■  .1  seconds 


- No-acceleratlon  displacement 

- Accelerated  displacement 

Velocity  ■  500  KTAS  Acceleration  ■  7  Gs 

FIGURE  1  TARGET  DISPLACEMENT 


Based  on  the  data  In  Table  1,  a  time  period 
of  one- tenth  of  a  second  Is  adequate  as  the 
boundary  on  a  zero-acceleration  assunptlon. 

Before  the  scoring  algorithm  Is  used,  It  must  be 
determined  when  the  weapon  Is  within  one-tenth  of 
a  second  of  time  of  impact  or  closest  miss.  Also, 
It  must  be  determined  If  one-tenth  of  a  second  Is 
a  short  enough  time  period  for  the  no  acceleration 
assunptlon  on  the  weapon  projectile.  ASPT  uses 
two  methods  at  this  point.  Each  method  used  Is 
based  upon  the  weapon  Involved,  l.e.,  gun  or 
missile. 

Gun 

Scoring  with  the  gun  must  take  Into  account 
the  fact  that  a  large  number  of  bullets  can  be 
fired,  each  with  the  potential  to  hit  the  target. 
Evaluating  each  bullet  Individually  on  every 
Iteration  would  require  massive  amounts  of 
computation  time;  or  storing  the  plotted  flight 
path  of  each  bullet  would  require  a  large  memory 
capacity, 

ASPT  uses  an  Innovative  approach  to  this 
problem.  As  the  bullet  Is  fired,  a  bullet  time  of 
flight  until  Impact,  based  on  the  range  to  the 
target,  Is  determined.  This  time  of  flight  Is 
used  to  predict  a  new  target  position  at  that  time 
of  predicted  Impact  by  using  existing  target 
position,  velocity,  and  acceleration.  The  new 
target  position  Is  used  to  calculate  a  new  range, 
and  the  process  Is  repeated.  This  Iterative 
process  provides  an  accurate,  predicted  Impact 
point.  Based  upon  the  predicted  range,  a 
predicted  time  of  flight  until  bullet  Impact  Is 
found.  Using  this  time  of  flight  and  equations 
that  model  the  bullet  flight  characteristics, 
bullet  position  and  velocity  for  the  predicted 
Impact  are  found.  These  values  (time  of  flight, 
Pox,  Poy,  Poz,  Vx,  Vy,  Vz)  are  stored  as  a  row  in 
an  array.  If  the  predicted  range  Is  greater  than 
the  effective  range  of  the  gun,  It  Is  assumed  the 
bullet  will  miss,  and  no  data  Is  stored  In  the 
array.  It  Is  Interesting  to  note  that  any  form  of 
ballistic  model  may  be  used  as  long  as  bullet 
position  and  velocity  can  be  determined  at  the 
predicted  Impact  point.  Another  Interesting  point 
Is  that  the  rate  of  fire  on  most  aircraft  cannons 
Is  greater  than  most  simulation  Iteration  rates. 
For  example,  the  A-10  30  mm  cannon  fires  at  70 
rounds  per  second  and  the  F-16  20  mn  cannon  fires 
at  100  rounds  per  second.  Therefore,  It  must  be 
assumed  that  each  bullet  value  that  Is  stored  Is 


not  one  bullet  but  actually  a  stream  of  bullets. 
The  values  stored  are  the  values  for  the  midpoint 
In  the  stream. 

One  of  the  data  values  stored  Is  time  of 
flight.  This  time  of  flight  Is  used  to  determine 
when  to  apply  the  scoring  algorithm.  One 
Iteration's  worth  of  time  (based  upon  the 
simulation  used)  Is  subtracted  from  the  time  of 
flight  value  on  each  Iteration.  The  time  of 
flight  remaining  Is  then  checked  to  see  If  It  Is 
within  one  half  of  an  iteration  of  zero.  When  the 
time  elapses  to  within  one  half  of  an  Iteration  of 
zero,  that  row  of  the  array  Is  evaluated.  By 
using  the  stored  bullet  position  and  velocity  and 
the  actual  target  position  and  velocity  In  the 
scoring  algorithm,  a  miss  distance  can  be 
determined. 

The  scoring  algorithm  is  based  upon  knowing 
the  weapon's  and  target's  position  and  velocity  at 
the  same  point  In  time.  But,  It  Is  unlikely  the 
time  of  flight  would  be  an  exact  multiple  of  the 
.  Iteration  rate  (causing  time  of  flight  to  subtract 
to  exactly  zero).  This  means  the  weapon's  and 
target's  position  and  velocity  values  are  probably 
not  for  the  same  point  In  time.  However,  the  time 
difference  would  be  small,  less  than  one  half  of 
an  Iteration.  This  dilemma  can  be  resolved  by 
referring  back  to  the  assumption  about  the  bullets 
being  a  stream.  The  velocity  at  any  position 
along  that  stream  can  be  considered  constant  for  a 
short  time  period.  Therefore,  the  bullet  position 
and  velocity  values  can  be  assuned  to  be  valid  at 
the  time  used  to  determine  the  target  position  and 
velocity.  (See  figure  2.) 


Position  and  velocities 
used  In  scoring  algorithm 


Actual  position  and  velocities 
when  scoring  algorithm  Is  applied 


Bullet  stream  and  velocity 


Target  position  and  velocity 


Hit  occurs  In  both  cases  but  at  different 
points  In  the  bullet  stream. 

FIGURE  2  BULLET  AND  TARGET  POSITION  AT  IMPACT 


After  a  row  of  the  array  has  been  evaluated, 
it  Is  zeroed  out  and  not  checked  again  unless  It 
Is  reused  with  the  values  for  a  new  bullet.  By 
using  this  method,  only  the  bullets  that  were 
predicted  to  hit  the  target  on  a  given  Iteration 
will  be  evaluated  on  that  Iteration.  Therefore, 
each  bullet  does  not  need  to  be  evaluated  or.  each 
iteration;  however  more  than  one  bullet  m*y  be 
evaluated  on  a  particular  Iteration. 
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The  ballistic  equations  that  account  for 
bullet  flight  need  only  be  evaluated  once  for  each 
bullet.  These  equations  are  run  at  the  time  of 
firing,  and  they  provide  the  position  and  velocity 
values  at  the  predicted  Impact.  Massive  memory  Is 
not  required  since  bullet  position  and  velocity 
are  saved  for  only  one  Instant  of  time.  The 
memory  required  on  ASPT  Is  a  local  array  60  X  7  In 
size  (32  bit  words).  The  "60"  dimension  Is  based 
upon  ASPT's  30  Hertz  (Hz)  iteration  rate  and  the 
fact  that  the  maximum  effective  range  of  the 
bullet  Is  always  reached  within  two  seconds; 
therefore,  only  60  rows  would  be  needed  at  any  one 
time.  Also,  this  method  should  ensure  the  bullet 
Is  within  one-tenth  of  a  second  of  Impact  on  the 
target.  Since  the  maximum  time  of  flight  for  the 
bullets  is  two  seconds,  (based  upon  effective 
range  at  the  gun)  It  Is  Impossible  for  a  target  to 
maneuver  In  such  a  way  that  It  could  pass  through 
the  bullet  flight  path  with  a  potential  for  being 
hit  without  being  quite  close  to  the  predicted 
Impact  point.  (See  figure  3). 


-  —  --—Bullet  flight  path 

—  —  —  —  Predicted  target  path 

. Actual  target  path 

O  Target  position  at 
predicted  impact 

FIGURE  3  POSSIBLE  BULLET  AND  TARGET  POSITIONS 


Now  that  the  time  to  apply  the  scoring 
algorithm  has  been  determined,  the  question 
remains  how  short  a  time  period  (based  upon  bullet 
ballistics)  Is  required  for  the  no  acceleration 
assumption  to  be  valid. 

There  are  two  primary  forces  which  cause 
accelerations  that  affect  a  bullet's  flight  path. 
The  first  force  Is  gravity,  which  acts 


perpendicular  to  the  earth's  surface.  Over  a 
short  time  period  (one-tenth  of  a  second),  the 
gravity  effect  can  be  disregarded.  The  second 
force  is  supersonic  drag,  which  acts  parallel  to 
the  bullet  flight  path.  Table  2  In  the  Appendix, 
shows  the  effects  of  drag  on  bullet  displacement 
and  velocity  over  a  1.5  second  time  period. 

Using  the  .5  sec  line  from  Table  2  as  an 
example,  we  find  that  at  .5  seconds,  bullet 
velocity  is  3140  ft/sec.  The  distance  the  bullet 
has  traveled  when  using  ballistic  equations  that 
account  for  drag  Is  1725  feet.  If  no  acceleration 
had  been  assumed  during  the  previous  one-tenth  of 
a  second,  the  distance  traveled  would  be  1734 
feet.  The  difference  of  nine  feet  Is  not 
significant  If  we  remember  that  the  bullet  Is 
actually  a  stream  of  bullets  extending  In  front  of 
and  behind  the  bullet  point  (See  figure  2). 
Therefore,  when  the  weapon  used  is  a  gun,  the 
one-tenth  of  a  second  time  period  Is  short  enough 
to  assume  no  acceleration  based  upon  the  data  In 
Table  2. 

The  last  point  to  be  addressed  In  this 
section  Is  how  the  one-tenth  of  a  second  time 
period  Is  used  vrfien  applying  the  scoring 
algorithm.  Recall  that  the  scoring  algorithm  Is 
applied  when  the  time  of  fll^it  to  the  predicted 
Impact  point  has  been  reduced  to  zero.  When  the 
scoring  algorithm  is  applied,  the  first  thing 
found  is  the  time  when  the  minimum  miss  distance 
will  occur.  This  time  Is  limited  to  an  absolute 
value  of  one- tenth  of  a  second,  the  time  period 
for  the  no -acceleration  assumption.  This  means 
the  bullet  must  penetrate  the  target  sphere  within 
one-tenth  of  a  second  from  the  time  of  the 
predicted  impact  if  it  is  to  be  recorded  as  a  hit. 

Missile 

missile  presents  a  slightly  different 
problem.  Since  the  missile  Is  often  fired  at  long 
range,  the  target  aircraft  will  have  time  to 
perform  some  maneuver  that  will  cause  it  to  be 
significantly  displaced  from  its  predicted 
position.  Also,  because  of  its  maneuvering 
capability,  the  missile's  flight  path  Is  unknown 
ahead  of  time.  However,  only  a  few  missiles  can 
be  active  at  one  time,  so  on  ASPT,  each  missile 
can  be  updated  on  every  Iteration  without 
requiring  excessive  execution  time  and  memory 
capacity. 

Since  both  the  missile  and  target  are 
maneuvering,  it  is  impossible  to  set  a 
predetermined  time  to  evaluate  the  miss  distance. 
Therefore,  a  different  technique  is  used.  The 
range  between  missile  and  target  is  determined  for 
each  iteration.  Once  the  missile  has  reached  its 
maximum  velocity.  If  the  range  between  successive 
Iterations  decreases,  the  missile  will  be  allowed 
to  continue  tracking  the  target.  As  soon  as  the 
range  Increases,  it  Is  assumed  the  missile  just 
passed  the  target.  Therefore,  on  the  first 
Iteration  that  shows  an  increase  in  range,  the 
scoring  algorithm  Is  applied  using  current  target 
and  missile  positions  and  velocities.  The  scoring 
algorithm  then  becomes  the  method  used  to 
Interpolate  between  target  and  missile  positions 
on  the  current  iteration  and  the  preceding 
Iteration.  ASPT  runs  on  a  30  Hz  Iteration  rate, 
therefore,  the  maxlmun  time  period  that  no 
acceleration  Is  assumed  Is  one-thirtieth  of  a 
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second  (one  Iteration).  This  time  Is  well  within 
the  one- tenth  of  a  second  time  period  that  was 
established  for  the  target.  The  only  possible 
problem  with  the  no-acceleratlcn  assumption  on  the 
scoring  algorithm  would  be  the  missile's  own 
maneuvering  capability.  Table  3,  in  the  Appendix, 
shows  the  effects  of  airspeed  and  G-loadlngs  on 
missile  displacement  for  a  one-thirtieth  of  a 
second  time  period.  Even  for  the  worst  case  of  a 
missile  traveling  2000  knots  with  the  capability 
to  pull  30  Gs,  the  displacement  due  to 
acceleration  over  one-thirtieth  of  a  second  is 
only  .536  feet.  Based  on  Table  3,  the 
no-acceleration  assunptlon  Is  valid  for  the 
missile  when  the  time  period  Is  one-thirtieth  of  a 
second  or  less.  The  procedure  for  a  missile, 
then.  Is  to  update  both  the  target  and  missile 
positions  on  every  Iteration.  When  the  missile 
stops  converging  on  the  target  and  first  shows  an 
increase  In  range,  the  scoring  algorithm  Is 
applied  using  current  values  of  position  and 
velocity.  The  maximum  time  for  the 
no-acceleration  assunptlon  then  becomes  one 
thirtieth  of  a  second. 

CONCLUSION 

The  ASPT  method  of  scoring  against  a  movable 
target  allows  real-time  scoring  using  actual 
target  and  weapon  positions.  This  method  requires 
a  minimal  amount  of  computer  memory  and  execution 
time  while  producing  a  high  fidelity  means  of 
determining  weapon  effectiveness  against  mobile 
targets. 


APPENDIX  A 
DISPLACEMENT  TABLES 


TABLE  1 

TARGET  DISPLACEMENT  FOR  VARIOUS  TIMES 


Airspeed 

(Knots) 

Acceleration 

(Gs) 

Time  Period 
(Sec) 

Displacement 

(Feet) 

300 

5 

.05 

.201 

300 

5 

.10 

.804 

300 

5 

.15 

1.809 

500 

7 

.05 

.282 

500 

7 

.10 

1.126 

500 

7 

.15 

2.534 

700 

9 

.05 

.362 

700 

9 

.10 

1.448 

700 

9 

.15 

3.257 

900 

11 

.05 

.442 

900 

11 

.10 

1.769 

900 

11 

.15 

3.981 

Displacement  is  measured  In  feet.  It  Is  the 
lateral  difference  between  where  the  zero- 
acceleration  velocity  vector  would  put  the  aircraft 
during  the  time  period,  and  where  the  accelerated 
velocity  vector  would  put  the  aircraft  during  the 
same  time  period. 


TABLE  2 

BULLET  DISPLACEMENT  AND  VELOCITY 


Time 

of 

Flight 

(Sec) 

Velocity 

(Ft/Sec) 

Distance 
Traveled  With 
Acceleration 
(Feet) 

Distance 

Traveled  Without 
Acceleration 
(Feet) 

0 

3806* 

0 

0 

.1 

3678 

373.2 

380.6 

.2 

3542 

731.2 

741.0 

.3 

3404 

1076 

1085 

.4 

3270 

1407 

1416 

.5 

3140 

1725 

1734 

.6 

3014 

2030 

2039 

.7 

2894 

23  23 

2331 

.8 

2777 

2604 

2612 

.9 

2664 

2874 

2882 

1.0 

2555 

3133 

3140 

1.1 

2450 

3381 

3389 

1.2 

234  7 

3619 

3626 

1.3 

2248 

3846 

3854 

1.4 

2152 

4064 

4071 

1.5 

2058 

4273 

4279 

*Muzzle  velocity  plus  true  airspeed. 


This  data  is  based  upon  the  F-16  20MM  cannon; 
the  data  Is  valid  for  ar  aircraft  at  10,000  MSL  and 
300  KTAS.  The  maximum  effective  range  of  this  gun 
Is  4000  feet.  The  "Distance  Traveled  Without 
Acceleration"  col  win  Is  based  upon  no  acceleration 
during  the  previous  one-tenth  of  a  second,  not  for 
the  entire  time  of  flight. 
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TABLE  3 

MISSILE  DISPLACEMENT 
OURING  ONE  THIRTIETH  OF  A  SECOND 


REFERENCES 


Airspeed 

(Knots' 

Acceleration 

(Gs) 

Displacement 

(Feet) 

500 

15 

.268 

500 

20 

.357 

500 

25 

.447 

500 

30 

.536 

1000 

15 

.268 

1000 

20 

.357 

1000 

25 

.447 

1000 

30 

.536 

1500 

15 

.268 

1500 

20 

.357 

1500 

25 

.447 

1500 

30 

.536 

2000 

15 

.268 

2000 

20 

.357 

2000 

25 

.447 

2000 

30 

.536 

Displacement  Is  measured  In  feet.  It  Is  the 
lateral  difference  between  where  the  zero- 
acceleration  velocity  vector  would  put  the  missile 
during  the  time  period,  and  where  the  accelerated 
velocity  vector  would  put  the  missile  during  the 
time  period. 


1.  Greenwood,  Donald  T.  Principles  of  Dynamics. 
Englewood  Cliffs,  New  Jersey:  Prentlce-nall , 
Inc.,  1965. 

2.  Hlbbeler,  R.  C.  Engineering  Mechanics: 
Statics  and  Dynamics.  New  Yorlc:  Macmillan 
Publishing  Co.,  Inc.  1974. 

3.  Technical  Order  1A-10A-1,  A-10A  Fllcftt 
Manual.  1  July  1979. 

4.  Technical  Order  1A-16A-1,  F-16A^>  Flight 
Manual .  22  August  1980. 

5.  Technical  Order  1A-16A-34-1-1,  Nonnuclear 
Munitions  Delivery,  F-16A  and  F-16B  Aircraft. 

9  January  1981. 

6.  Technical  Order  1A-16A-34-1-2,  Nonnuclear 
Munitions  Delivery  Ballistics.  F-16A  and  E-16B 
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THE  M58A1  TRAINING  AID,  PERSONAL  DECOMTAMINATION  KIT 


Mr.  James  M.  Szachta 
Project  Engineer 
U.  S.  Army  ARRADCOM 
Aberdeen  Proving  Ground,  Maryland 

ABSTRACT 


The  M2S8A1  Kit  Is  a  personal  decontamination  system  which  provides  to 
the  individual  soldier  the  capability  of  decontaminating  skin  areas  exposed 
to  chemical  agents.  The  M58A1  Kit  is  specifically  designed  for  training  troops 
in  the  use  of  the  M258A1  Kit.  The  original  design  of  the  kits  contained  short¬ 
comings  and  deficiencies:  (1)  difficulty  in  the  preparation  of  the  chemicals 
fer  use;  (2)  lack  of  a  facial  decontamination  capability;  (3)  availability  of 
only  one  complete  decontamination  per  kit;  (4)  unreasonable  amount  of  training 
required  for  proficiency.  Studies  indicated  that  these  deficiencies  could  be 
overcome  by  repackaging  the  chemicals  into  a  towelette  configuration  hermeti¬ 
cally  sealed  in  a  laminated  plastic-foil  packet.  A  product  Improvement  effort 
was  initiated  and  the  product  Improved  kit  had  the  following  benefits:  (1) 
safer,  easier,  and  faster  to  use;  (2)  operational  equivalence  to  the  original 
kit  plus  partial  facial  decon  and  the  capability  to  decontaminate  personal 
equipment;  (3)  three  complete  decontamination  systems  per  kit; 

(4)  reduced  costs. 


INTRODUCTION 

Early  and  effective  decontamination  of  toxic 
chemical  agents  on  skin  can  prevent  loss  of 
combat  effectiveness  or  preclude  death.  Therefore 
factors  that  expedite  onset  of  decontamination 
are  as  relevant  as  rates  of  chemical  decontamina¬ 
tion  or  differences  of  cleansing  methods.  Testing 
conducted  by  the  U.S.  Army  Biomedical  Laboratory 
has  shown  that  it  was  more  important  to  get  the 
decon  solutions  on  the  akin  in  order  to  neutralize 
the  agent  effects  than  to  attempt  to  physically 
remove  the  agent.  In  April  of  1980,  the  function 
of  the  personal  decontamination  kit  was  expanded 
to  Include  the  decontamination  of  personal  equipment 
items  such  as  the  M-16  rifle  and  rubber  gloves. 

This  report  summarizes  a  program  to  field  an 
improved  decontamination  kit  which  is  substantially 
easier  to  use,  and  considerably  faster  to  deploy. 

The  M258  Decontamination  Kit 

The  M258  kit  is  a  personal  decontamination 
system  containing  materials  to  allow  the  individual 
soldier  to  decontaminate  skin  areas  exposed  to 
chemical  agents.  The  M58  is  the  training  kit. 

The  M58  training  kit  is  identical  to  the  M258 
decon  kit  except  the  active  chemicals  have  been 
replaced  by  inactive  simulants;  the  chemicals 
in  Solutions  I  and  II  are  replaced  with  a  502 
propanol  and  water  solution,  and  the  chloramine 
B  is  replaced  by  sodium  chloride  crystals.* 

The  M258  decontamination  kit  is  composed  of  the 
following: 

'  four  gauze  pads 

•  two  plastic  sticks 

•  two  plastic  capsules  containing  the  active 
decontaminating  solution 

•  a  water  proof  plastic  container  to  package 
the  above  items. 


*For  clarity,  we  will  generally  restrict  ourselves 
to  discussing  the  M258  decon  kit,  however,  identi¬ 
cal  Improvements  of  the  M58  training  kit  are 
implied . 


The  number  I  capsule  of  decontaminating 
solution  (solution  I),  capable  of  decontaminating 
GD  and  thickened  GD,  contains  40  ml  of: 

•  sodium  hydroxide  52 

•  phenol  102 

•  ammonia  0.22 

•  hydroxy  ethane  722 

•  water  remainder. 

The  number  II  capsule  of  decontaminating 
solution  (solution  II),  capable  of  decontaminating 
HD  (mustard)  and  VX  agents,  contains  53  ml  of: 

•  hydroxy  ethane  442 

•  zinc  chloride  52 

•  deionized  water,  remainder 

•  a  sealed  glass  ampoule  containing  Chlora¬ 
mine  B,  17  gm. 

Use  of  the  M258  Kit 

A  step-by-step  procedure  is  specified  for 
use  of  the  M258  kit:  (1)  open  the  decon  kit; 

(2)  use  a  gauze  pad  from  the  kit  to  soak  up  any 
liquid  contamination  that  is  on  the  skin;  (3) 
if  the  contamination  is  sticky  or  greasy  and 
will  not  soak  up  into  the  gauze,  the  red  plastic 
scrapers  are  used  to  scrape  the  contamination 
away;  (4)  using  the  spike  on  the  container  cover, 
punch  a  hole  in  the  side  of  capsule  I;  (5)  soak 
a  fresh  gauze  pad  with  the  decon  solution  from 
capsule  I  and  swab  or  wipe  contaminated  areas 
of  skin  with  solution  I  wetted  pad  for  one  minute; 
(6)  break  the  glass  ampoule  that  is  Inside  capsule 
II  by  hitting  the  wide  side  of  the  capsule  with 
a  solid  blow  against  a  hard  surface  (boot  heel, 
weapon,  rock,  etc.);  (7)  shake  capsule  II  hard 
twelve  times  so  that  contents  are  well  mixed; 

(8)  punch  a  hole  in  the  side  of  capsule  II  using 
the  spike  on  the  container  cover;  (9)  soak  a 
fresh  gauze  pad  with  decon  solution  from  capsule 
II  and  then  swab  or  wipe  contaminated  areas  of 
skin  with  the  solution  II  wetted  pad  for  two 
to  three  minutes. 
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Shortcoming!  and  Deficiencies  of  the  M258  Kit 


PRODUCT  IMPROVEMENT  PROGRAM 


The  present  M258  kit  contains  several  short¬ 
comings  and  deficiencies:  (1)  Inadvertent  ampoule 
tip  breakage  (leading  to  rapid  breakdown  of  chlora¬ 
mine  B  and  loss  of  decontamination  capability;  (2) 
difficulty  of  breaking  the  glass  ampoule  lnslae 
capsule  II;  (3)  knowing  under  dark  conditions 
whether  glass  ampoule  Is  broken;  (4)  Inadvertent 
breaking  of  capsule  II  while  attempting  to  break 
the  glass  ampoule,  leading  to  a  loss  of  decontamina¬ 
tion  capability;  (5)  spike  in  container  lid  (used 
for  breaking  the  plastic  capsules)  can  break 
off  or  can  cause  damage  to  rubber  glove  or  injury; 

(6)  uncertainty  under  dark  and  gloved  conditions 
whether  capsule  was  broken  with  the  spike  and 
whether  the  gauze  pad  was  properly  wetted;  (7) 
no  capability  for  facial  decontamination;  (8) 
only  one  complete  decontamination;  (9)  unreason¬ 
able  amount  of  training  time. 

Improving  the  M258  Kit 

Mine  Safety  Appliances  Company  (MSA)  under 
contract  to  the  Chemical  Systems  Laboratory  (CSL), 
had  a  one-year  effort  (1977-1978)  to  Investigate 
Improving  the  decontamination  kit.  The  objectives 
of  the  program  were  to  prove  the  feasibility 
of  using  the  components  of  the  M258  kit  (Decon 
I  and  Decon  II  solution)  on  equipment;  and  design 
a  suitable,  compact.  Individual  package/dispenser 
for  these  solutions  using  compatible  component 
materials,  and  demonstrate  the  feasibility  and 
utility  of  this  design. 

An  experimental  program  was  conducted  which 
examined  packaging  materials;  and  performed  shelf- 
life  tests,  storage  tests,  permeation  tests; 
and  the  effects  of  temperature,  decontamination 
procedures,  and  decontamination  tests  with  candi¬ 
date  materials.  Finally,  a  design  study  culminated 
in  ten  proposed  prototype  kits. 

The  following  conclusions  resulted  from  this 
program.  (1)  Capsule  I  from  the  M258  kit  could 
be  repackaged  in  a  foil  pouch  with  a  premoistened 
wipe.  This  concept  is  similar  to  the  individual 
moistened  hand  wipes  used  by  the  airlines,  etc. 

(2)  Capsule  II  from  the  M258  kit  could  be  repackaged 
in  a  foil  pouch  with  a  chloramine  B  impregnated 
wipe.  The  alcohol/water  solution  is  delivered 
to  the  wipe  from  a  single  crushable  glass  ampoule 
which  is  also  in  the  pouch  and  which  is  encapsulated 
in  100-mesh  propylene  screen  that  effectively 
retains  100X  of  the  glass.  (3)  Repackaging  the 
components  of  the  M258  kit  in  foil  pouches  provides 
a  kit  which  is  equal  or  better  in  decon  efficiency, 
and  simpler  and  quicker  to  use  than  the  M258 
components.  (4)  The  feasibility  of  utilizing 
the  decontaminating  solutions  (solutions  I  and 
II)  on  equipment  was  demonstrated. 

This  Joint  effort  between  CSL  and  MSA  devel¬ 
oped  the  basic  information  for  a  product  improvement 
program  (PIP).  This  effort  provided  data  on 
packaging  concepts  and  established  the  ultimate 
selection  of  a  wipe  material  and  laminated  foil 
packaging  material,  the  safety  of  a  mesh-encased 
glass  ampoule,  and  the  material  compatibility 
testing. 


Objective 

To  correct  the  shortcomings  and  deficiencies 
of  the  M258  kit,  the  Chemical  Systems  Laboratory 
(CSL)  commenced  with  a  product  improvement  program 
(PIP).  The  objective  of  this  program  was  to 
correct  deficiencies  in  the  M258  Decon  Kit  and 
M58  Training  Aid  Kit  by  repackaging  the  existing 
chemicals  into  a  towelette  configuration. 

The  actual  PIP  effort  was  initiated  in  fiscal 
year  1979,  and  was  to  take  approximately  20  months 
to  complete.  A  meeting  with  the  Army  Armament 
Material  Readiness  Command  (ARRCOM)  in  April 
1979  resulted  in  an  expedited  effort  to  implement 
the  benefits  of  the  product  improved  kit  into 
a  procurement  action  directed  to  purchase  1,000,000 
kits.  The  expedited  effort  resulted  in  shortening 
the  schedule  by  six  months. 

Benefits 

The  product  improved  kit  has  the  following 
benefits:  (1)  safer,  easier,  and  faster  to  use; 

(2)  operationally  equivalent  to  the  current  kit 
plus  a  facial  decon;  (3)  three  complete  decontam¬ 
inations;  (4)  substantially  reduced  costs  through 
more  efficient  and  effective  training.  It  gives 
three  times  the  capability  at  approximately  the 
same  cost. 

Minimum  Requirements 

The  U.S.  Army  Training  and  Doctrine  Command 
(TRADOC)  Imposed  minimum  requirements  for  the 
product  Improved  kit.  These  specific  requirements 
were:  (1)  the  kit  must  be  placed  in  a  box  which 
easily  fits  the  product  into  the  M17  and  XM29  car¬ 
rying  cases;  (2)  the  kit  muse  have  a  capability  of 
face  decontamination;  (3)  the  kit  must  contain  the 
maximum  amount  of  skin  decontamination  towelettes 
(solutions  I  and  II,  and  facial  decons)  which 
can  be  fitted  into  the  case;  (4)  the  title  is 
to  read  "Improved  Personal  Decontamination  Kit". 
(This  last  requirement  has  since  been  waived 
as  It  was  a  criterion  having  nothing  to  do  with 
functional  capability.) 

Tasks  for  PIP 

The  CSL  conducted,  supervised  and  coordinated 
the  product  improvement  program.  Five  tasks 
were  identified: 

(1)  finalization  of  kit  design; 

(2)  biomedical  testing; 

(3)  environmental  testing; 

(4)  Human  Engineering  Laboratory  (HEL) 
testing; 

(5)  preparation  of  a  Technical  Data  Package 
(TDP) . 

A  contract  was  negotiated  with  Batlelle 
Columbus  Laboratories  (BCL)  to  assist  in  tasks 
(1),  (3),  and  (5). 

Finalization  of  Kit  Dealgn 

The  objectives  of  this  task  were  to  enhance: 

(1)  night  identification; 

(2)  packet  design  and  guide  tear  feature; 

(3)  packet  materials  and  marking; 

(4)  ampoules; 

(5)  plastic  mesh  screen; 
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(6)  towlettes; 

(7)  containers; 

(8)  container  labels; 

(9)  prototype  PIP  components  and  kits. 

Night  Identification.  A  method  was  needed 
whereby  the  soldier  could  distinguish  between 
the  decontamination  packets  1  and  2.  The  soldier 
must  be  able  to  make  this  Identification  at  night 
without  the  use  of  lights  with  or  without  gloves. 

The  fastest  and  surest  Identification  may 
be  anticipated  by  the  use  of  large  features  or 
configurations.  Thus,  the  fact  that  the  decon 
1  packet  Is  thin  and  soft  (or  pliable)  and  the 
decon  2  packet  thick  and  stiff  (because  of  the 
glass  ampoules)  Is  one  of  the  best  distinguishing 
features  built  directly  into  the  PIP  version 
of  the  kits.  The  only  drawback  stems  from  the 
fact  that  identification  can  only  occur  after 
withdrawal  of  the  packet  from  the  holder  (which 
is  also  the  case  with  most  other  Identification 
schemes) . 

An  additional  identification  feature  proposed 
by  CSL  and  Incorporated  into  the  packets  orepared 
at  Battel le  were  tabs  at  the  top  of  the  packets. 
These  tabs  could  serve  for  initial  identification 
before  the  packets  are  reraoved  from  the  holder 
and  could  also  facilitate  removal,  especially 
when  rubber  gloves  are  being  worn.  A  number 
of  variations  of  these  Identification  tabs  were 
explored,  and  finally  tabs  of  1 .27-cm  (1/2-inch) 
base  and  0.79-cm  (5/16-lnch)  height  were  adopted. 
Higher  tabs,  because  they  must  be  bent  over  to 
close  the  lid  of  the  container  might  interfere  with 
the  water  tight  seal  if  one  got  caught  underneath 
the  lid  gasket.  Shorter  tabs  would  cause  removal 
and  Identification  problems  with  gloved  fingers. 
(One  of  the  changes  made  in  labeling  was  to  change 
the  roman  numerals  I  and  II  to  the  arable  numerals 
1  and  2.  We  shall  be  consistent  throughout  the 
report  in  referring  to  the  original  M258  kit 
decon  solutions  with  a  I  and  II,  and  the  PIP 
packets  with  a  1  and  2.) 

The  other  concern  that  demanded  much  attention 
was  the  placement  of  the  tabs  on  the  top  edge 
of  the  packets.  Configuration  A,  Figure  1,  shows 
the  position  of  1  and  2  tabs,  respectively,  for 
decon  1  and  2  packets  at  the  corner  of  each  packet, 
as  they  are  to  be  loaded  into  the  plastic  container. 
Other  configurations,  such  as  Indenting  the  tabs 
1.27  cm  (1/2  inch)  from  each  corner  (B  in  Figure  1), 
separating  the  two  tabs  in  opposite  corners  (C 
In  Figure  1),  or  providing  one  tab  in  the  middle 
of  one  packet  and  no  tabs  on  the  other  packet 
(D  in  Figure  1)  were  rejected  as  leading  to  confu¬ 
sion  under  emergency  conditions.  Configuration  A 
was  considered  to  be  the  most  acceptable  design. 
Subsequent  efforts  by  MSA  and  CSL  to  optimize 
the  mass  production  manufacturing  producibility 
of  the  packets  resulted  In  Configuration  D  for 
the  packets.  The  two-tab  design  did  not  lend 
Itself  to  existing  mass  production  techniques. 

Final  design  criteria  was  approved  on  March  25, 

1980,  at  a  meeting  involving  TRADOC,  MSA  and 
CSL.** 


**In  January  1980,  MSA  had  been  awarded  a  produc¬ 
tion  contract  from  ARRCOM  to  produce  3,000,000 
kits.  Thus,  MSA  was  Involved  In  the  final 
design  from  the  point  of  view  of  mass  production. 


The  single  tab  available  on  the  Decon  1 
packet  serves  a  dual  purpose.  It  serves  as  an 
aid  for  night  identification  when  the  user  Is 
using  rubber  gloves,  and  Is  a  practical  means 
for  removing  the  initlnl  packet  from  the  kit  con¬ 
tainer.  When  this  one-tab  and  no-tab  configuration 
was  selected,  TRADOC  concluded  that  the  previously- 
mentioned  concern  for  the  confusion  that  could 
arise  from  the  use  of  these  packets  would  not 
constitute  a  real  problem.  Identification  of  the 
packets  could  be  solved  through  proper  personnel 
training. 

Packet  Design  and  Quick  Tear  Feature.  In 
addition  to  the  tab  design,  several  decisions 
needed  to  be  made  to  arrive  at  a  final  packet 
configuration.  These  Included  the  rounding  of 
corners  and  the  placement  of  notches  to  aide 
in  Initiating  the  tearing  open  of  the  packets. 

The  rounding  of  all  corners  was  done  early 
in  the  program  including  the  outside  corners 
of  the  tabs.  However,  it  was  decided  that  the 
tabs  would  be  more  functional  with  the  corners 
not  rounded.  The  remaining  corners  were  rounded 
to  0.64-cm  (1/4-inch)  radius  to  accomplish  easier 
loading  into  the  containers,  to  obtain  a  better 
fit  with  the  rounded  bottoms  of  the  containers, 
and  to  avoid  Injury  to  exposed  skin  from  the 
sharp  corners. 

Emphasis  was  then  directed  to  provide  a 
quick-tear  opening  mechanism.  A  major  advantage 
of  the  PIP  version  of  the  kits  is  that  the  soldier 
In  the  field  can  obtain  faster  skin  decontamination 
because  of  greater  simplicity  and  faster  access 
to  the  materials.  The  quick-tear  open  feature 
of  the  packets  Is  an  Important  aspect  of  their 
design.  A  variety  of  different  notch  placements, 
shapes  and  dimensions  were  investigated  in  conjunc¬ 
tion  with  the  various  tab  configurations  shown 
earlier. 

The  Initial  design  efforts  by  MSA  resulted 
In  notches  placed  differently  on  the  PIP  1  and 
PIP  2  packets.  The  discussion  to  follow  describes 
the  progression  of  the  qulck-tear  design  evolving 
from  the  work  performed  by  MSA  in  their  original  con¬ 
tract  (1977-78).  Tab  design  progressed  concurrently. 
As  shown  in  Figure  2A,  PIP  1  packets  were  notched 
for  opening  at  the  sides  and  the  PIP  2  packets 
required  opening  from  the  top  or  bottom.  This  de¬ 
sign  introduces  two  possible  sources  of  deployment 
delay.  It  is  apparent  that  the  user  may  lose 
time  while  rotating  packets  into  two  different 
positions  at  90°  to  each  other;  but  also  confusion 
might  arise  In  distinguishing  between  notches 
up  or  down  (PIP  2)  and  left  or  right  (PIP  1), 
especially  at  night  and/or  gloved.  A  variety  of 
different  notch  placements,  shapes  and  dimensions 
were  then  investigated  in  conjunction  with  the 
various  tab  configurations. 

As  already  discussed,  CSL  design  contributions 
Introduced  the  addition  of  pull  tabs  to  facilitate 
identification  and  removal  of  packets  from  the 
case.  Ap  shown  In  Figure  2B,  for  the  Initial 
CSL  design,  notches  were  cut  adjacent  to  the 
PIP  1  tab  and  between  the  two  tabs  on  the  PIP 
2  packet. 

Observations  during  initial  human  factors 
tests  (discussed  later  In  this  report)  Indicated 
that  the  test  participants  were  experiencing 
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1  Decon  Packet  Decon  Packet 


Figure  2.  Development  of  Tear  Notches  and  Identification 
Tab  Design 


difficulties  with  the  opening  of  the  packets. 

Often  the  corner  of  the  packets,  shown  in  Figure 
2B,  were  torn  off  and  the  decontamination  pad 
could  not  be  removed  from  the  packets.  Configura¬ 
tion  of  the  tear  notches  was  such  that  the  corner 
of  the  packet  tended  to  tear  off  rather  than 
the  entire  side  of  the  packet. 

To  eliminate  the  problems  listed  above, 
an  improved  CSL  design,  shown  in  Figure  2C,  was 
then  introduced.  The  primary  notches  adjacent 
to  the  tabs  were  redesigned  to  face  inward  to 
affect  tearing  towards  the  middle  of  the  packet. 

In  addition,  partial  notches,  i.e.,  nicks  with 
the  apex  pointed  parallel  to  the  sides,  were 
introduced  at  the  opposite  corner  and  at  two 
locations  at  the  bottom  to  serve  as  a  secondary 


means  to  opening  the  packet  should  the  primary 
notches  fail .  Unfortunately,  occasional  tears  at 
the  corners  of  the  packets  were  still  experienced. 

At  the  previously  referenced  producibility 
design  meeting  (March  25,  1980),  this  problem 
was  presented  which  led  to  the  final  design, 
shown  in  Figure  2D.  The  notches  were  placed 
along  the  side  to  cause  tearing  with  the  grain 
of  the  laminated  foil  packet  material  and  also 
minimize  the  length  of  the  tear. 

Packet  Materials  and  Marking.  Candidate 
packet  materials  which  had  been  selected  In  the  prev¬ 
ious  MSA  effort  were  verified  through  permeation 
and  environmental  studies  that  are  reported  later 
in  this  report.  The  packets  consisted  of  laminated 
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retort  poach  notarial  of  0.5-mil  polyester  or 
nylon  outer  layer,  0.35-tail  aluminum  fell  middle 
layer  and  3  to  4-mil  modified  polyethylene  or 
polypropylene  inner  layer. 

Packet  marking  requirements  were  also  estab¬ 
lished  during  the  PIP.  Originally  an  extensive 
message  had  been  specified  for  each  side  of  the 
packets.  This  consisted  of  a  hazard  warning 
and  identification  on  one  side,  and  abbreviated 
instructions  for  use  on  the  other  side.  These 
messages  were  printed  on  polyester  labels  that 
were  affixed  to  tha  corresponding  packets.  It 
was  found  that  the  added  strength  of  the  polyester 
labels  inhibited  the  tearing  of  the  packets. 

Although  it  was  discovered  that  notching  the 
label  could  circumvent  this  problem,  it  was  decided 
to  eliminate  the  label  and  print  any  message 
directly  on  the  aluminum  foil  middle  layer  of  the 
packets.  The  reduction  of  thickness  (4  mil/package) 
aided  in  loading  and  removing  of  packets  from 
the  container.  As  mentioned  before,  it  was  also 
decided  that  the  roman  numbers  I  and  II  would 
be  changed  to  arable  numerals  1  and  2.  Results 
of  biomedical  data  and  Surgeon  General  Approval 
resulted  in  a  progression  of  these  messages. 

Ampoules.  The  decon  2  packet  serves  the 
dual  mission  of  neutralizing  the  alkalinity  left 
on  the  skin  from  the  use  of  the  dccon  1  wipe 
and  of  deactivating  remaining  threat  agents. 

Upon  recommendations  from  the  Biomedical  Laboratory 
and  CSL,  the  volume  of  fluid  in  the  decon  2  packets 
was  to  be  doubled  to  better  accomplish  these  dual 
objectives.  This  could  be  achieved  by  incorporat¬ 
ing  four  instead  of  two  small  glass  vials  into  the 
decon  2  packets.  Alternatively,  two  large  vials 
could  be  substituted  for  the  two  smaller  ones. 

Packets  containing  four  small  ampoules  of 
6.75-mm  diameter  each,  or  two  large  ampoules 
of  10.5-mm  diameter  each,  were  assembled  and 
submitted  to  CSL  for  evaluation.  It  was  decided 
to  equip  all  decon  2  packets  with  four  small 
ampoules  because  of  bettor  space  utilization  in 
the  plastic  container.  However,  during  evaluation 
tests,  it  was  determined  that  the  amount  of  solu¬ 
tion  available  was  far  In  excess  to  the  amount  that 
could  be  soaked  up  by  the  towelette.  This  excess 
solution  would  run  and  drip  and  possibly  preclude 
facial  use.  It  was  thus  determined  that  the 
optimum  saturation  of  the  towelettes  could  be 
obtained  through  the  use  of  three  ampoules. 

Plastic  Mesh  Screen.  The  plastic  mesh  screen 
serves  the  function  of  retaining  the  glass  from 
the  ampoules  after  crushing  and  not  letting  glass 
sp] Intern  injure  the  skin.  The  polypropylene 
mesh  selected  during  the  original  contract  with 
MSA  was  found  quite  adequate,  and  no  further 
development  work  was  done  cm  this  component. 

Towelettes.  An  80/20  rayon/ polypropylene 
nonwoven  fabric  had  been  selected  during  the  orig¬ 
inal  contract  with  MSA.  As  part  of  the  present 
development  effort,  the  size  of  the  towelette 
waa  to  be  increased  from  7.62  x  7.62  cm  (3  x 
3  Inches)  to  7.62  x  14.0  cm  (3  x  5-1/2  Inches). 

The  larger  towelette  can  be  held  flat  against 
the  palm  of  the  hand  and  secured  at  one  corner 
by  pressure  from  the  thumb.  This  permits  more 
efficient  decontamination  than  the  7.62  x  7.62 
cm  (3  x  3  inch)  swatch  that  tends  to  be  pinched 
between  the  thumb  and  the  finger.  The  larger 


towelette  also  supplies  a  larger  quantity  of 
chloramine  B  at  a  constant  loading  per  unit  area. 

Although  initial  trials  with  unimpregnated 
towelettes  had  indicated  that  there  was  sufficient 
space  in  the  packets  for  the  larger  towelettea 
and  that  three  sets  of  decon  1  and  decon  2  packets 
could  be  fitted  into  the  container,  the  total 
bulk  tended  to  Increase  considerably  following 
impregnation  of  the  towelettes.  The  fit  in  the 
container  was  excessively  tight  and  both  loading 
and  removal  were  difficult. 

To  alleviate  this  problem,  a  compromise 
in  towelette  size  was  adopted.  The  size  was 
reduced  to  7  x  12  cm  (2-3/4  x  4-3/4  Inches)  and 
folding  into  three  parts  instead  of  the  original 
four  parts  was  authorized  by  CSL.  Now  each  packet 
contributed  three  Instead  of  four  layers  of  towel¬ 
ettes  to  the  overall  thickness  of  the  assembly. 

This  produced  a  significant  improvement  of  the 
fit  of  the  packets  in  the  containers. 

Containers .  The  dimensions  of  the  polypropy¬ 
lene  containers  could  not  be  altered,  since  the  con¬ 
tainers  had  to  fit  into  the  space  of  existing 
mask  carriers.  Several  minor  modifications  of 
the  existing  containers  were  requested  by  CSL 
and  were  carried  out  at  Battelle.  These  included: 

(1)  removal  of  the  spike  from  the  lid  of 
the  containers; 

(2)  shearing  off  the  bulge  of  the  spike 
seat  in  the  lid  of  the  containers; 

(3)  heat  sealing  nylon  straps  on  both  sides 
of  the  mechanical  fastener. 

Container  Labels.  The  main  purpose  of  the 
container  labels  is  to  identify  the  contents 
and  to  Instruct  the  use  of  the  kit.  The  labels 
were  Jointly  arrived  at  by  CSL  and  Battelle. 

For  durability,  the  label  stock  selected  was 
2-mil  polyester  and  the  message  was  printed  on 
the  back  of  the  label  underneath  the  adhesive. 

The  label  was  matted  to  avoid  reflection  (from 
the  standpoint  of  readability  as  well  as  camouflage). 
Printing  with  a  chrome  yellow  ink  was  at  first 
specified,  but  since  the  message  appeared  weak 
against  the  dark  backgrounds,  a  gold  ink  was 
finally  specified. 

Prototype  PIP  Components  and  Kits.  Both 
PIP  modified  MJ58  and  M58  kits  were  prepared  by 
Battelle  for  standards  testing,  human  engineering 
evaluation,  display  kits,  etc.  Since  many  of 
the  methods  and  problems  associated  with  this 
small  scale  fabrication  will  be  different  for 
large-scale  manufacturing,  the  fabrication  will 
not  be  further , discussed ,  except  that  all  PIP 
modifications  essentially  used  "tried  and  proven" 
commercial/ Industrial  techniques,  and  no  major 
problems  were  anticipated  for  mass  production. 


Biomedical  Laboratory  Evaluations 


Objective.  The  U.  S.  Army  Biomedical  Labora¬ 
tory  at  Aberdeen  Proving  Ground,  Maryland,  conducted 
two  comparative  studies  for  the  CSL.  After  discus¬ 
sion  with  the  CSL,  the  comparative  requirements  were 
restated  as  the  following  two  experimental  studies: 


(1)  determine  the  relative  decontaminating 
efficacies  of:  (a)  Fuller's  earth  (M13  pad); 

(b)  alcohol  and  water  wipe  (M58  training  wipe 
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X);  (c)  PIP  decon  wipe  2j  (d)  sequential  une 
of  PIP  decon  wipe*  1  and  2; 

(2)  determine  the  relative  efficacies  for 
sequential  use  of  the  PIP  decon  wipes  l  and  2 
versus  sequential  use  of  M258  kit  components 
according  to  directions  for  both  liquid  and  sticky 
agents. 

Testing  Procedures.  Three  agents  [GD,  thick" 
ened  GD  (TGD) ,  and  VX] were  applied  on  clipped  backs 
of  rabbits  for  2.0  minutes  prior  to  initiation 
of  decontamination  procedures,  given  simulated 
field  conditions.  To  perform  the  aforementioned 
experimental  studies,  18  LD50  determinations 
were  required. 

Testing  required  the  use  of  rabbits.  The 
required  comparisons  could  not  be  performed  by 
in  vitro  methods  because  there  is  no  model  for 
duplication  of  interactions  between  cleansing 
methods,  reagents  and  toxic  agents  in  the  presence 
of  circulation  and  biochemistry  of  living  skin. 

The  rabbit  is  the  species  of  choice  because  clipped 
backs  of  rabbits  are  of  the  minimum  site  required 
for  use  of  wiping  materials  and  methods  of  decontam¬ 
ination  proposed  for  human  use.  The  value  of 
this  model  is  well  documented. 

Test  Results.  Mortality  fractions  were 
recorded  the  day  following  each  exposure.  Any 
animals  showing  severe  signs  were  held  to  complete 
a  period  of  24  hours  after  exposure  to  agent. 

Daily  parallel  exposure  of  rabbit  pairs  to  all 
experimental  variables  permitted  use  of  Thompson 
and  Well  analysis  for  determination  of  LD50  values. 
LD50  results  for  each  test  condition  were  compared 
with  control  results  to  produce  a  relative  protection 
factor  (the  control  results  are  assigned  a  value 
of  1.0;  a  number  larger  than  1.0  signifies  that  some 
protection  was  provided  by  the  "decontamination" 
procedure,  the  larger  the  number  the  more  effective 
the  procedure). 

The  results  of  the  Biomedical  Laboratory 
tests  are  summarized  in  Table  1.  The  information 
above  the  dashed  line  In  Table  1  can  be  interpreted 


as  the  relative  effectiveness  for  facial  decontam¬ 
ination.  The  overall  comparison  of  the  three 
facial  decontamination  options  suggests,  in  the 
absence  of  significant  differences  ags  GD 
and  TGD,  the  product  Improved  decon  2  superiority 
against  VX  gives  the  advantage  to  this  option. 
Likewise,  it  was  demonstrated  that  sequential 
use  of  the  PIP  solution  1  and  2  la  slightly  more 
effective  than  the  original  M258  kit. 

The  relatively  small  apparent  differences 
between  factors  for  dacontamlnanta  of  GD  and 
TGD  is  strikingly  different  from  the  large  factors 
and  differences  observed  with  VX  decontaminants. 

This  contrast  is  less  surprising  in  view  of  the 
ways  the  physical  propertlea  of  these  agents 
affect  their  control  LD50's.  GD  and  TGD  tend  to 
evaporate  from  akin  within  minutes  after  exposures; 
therefore,  a  substantial  portion  of  any  applied 
dose  will  have  evaporated  or  entered  the  skin 
within  two  minutes  after  exposure,  whether  or 
not  decontamination  is  performed  subsequently. 

[The  two-«inute  wait  to  deployment  was  part  of 
the  skin  decon  kit  deployment  scenario.]  For 
this  reason  decontamination  has  relatively  less 
effect  with  GD  than  VX,  which  persists  on  the 
skin  of  controls  long  after  decontamination  is 
carried  out  on  test  animals. 

Environmental  Testing 

MIL-STD-810C  Tests.  The  evaluation  of  the 
decontamination  kits  prepared  under  the  present 
program  included  the  performance  of  a  series 
of  physical  testB  and  of  accelerated  storage 
tests  of  three-month  and  six-month  duration. 

The  tests  were  conducted  on  the  PIP  kits  and 
packets  in  accordance  with  M1L-STD-810C.  Included 
in  these  tests  were  low  pressure  (altitude)  testing, 
low  temperature  testing,  high  humidity  testing, 
leakage  (immersion)  testing,  and  shock  (transit 
drop)  testing.  The  product  improved  kit  success¬ 
fully  passed  all  these  tests  with  a  confidence 
level  between  90  and  99X. 

Vibration  Tests.  Vibration  tests  were  per¬ 
formed  by  CSL.  Each  test  consisted  of  a  cycling 
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Table  1.  Biomedical  Test  Results:  Protection  Factors 


Protection 

Factors  for 

given  aqents** 

SB 

f  T5T5 

VX  factors 

Control 

1.0 

1.0 

1 

M58  Training  Wipe  I 

1.7* 

2.2 

56 

(Alcohol  and  Water) 

M13  Pad  (Fuller's  Earth) 

2.6* 

1.5* 

80 

PIP  Decon  Wipe  2 

2.2* 

4.0 

197 

SEQUENTIAL  WIPES 

M258  Kit 

3.6 

3.5 

90 

PIP  M2 58  Kit  (M258A1) 

3.9 

6.1 

96 

*  No  significant  difference  from  control  value. 
**  Based  on  clipped  rabbits. 
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of  the  room  temperature  control  and  of  the  storage 
bottle.  The  concentration  of  the  main  constituents 
of  solution  1  (phenol,  ammonia  (NH  ) ,  sodium 
hydroxide  (NaOH),  and  ethanol]  determined  by 
the  analyses  are  summarized  In  Table  3. 


test  during  which  there  was  a  series  of  sinusoidal 
sweeps  from  3  to  200  Hz  and  back  to  3  Hz  In  12 
minutes.  There  were  seven  sweeps  for  84  minutes 
for  each  axis  of  the  container.  Kits  were  vibrated 
at  a  high  temperature  of  approximately  43*0  and 
at  a  low  temperature  of  approximately  -34°C. 

Between  vibration  treatments  at  the  different 
temperatures,  the  packets  were  opened  and  examined 
for  breakage. 

With  one  failure  In  80  items,  the  decon 
2  packet  demonstrated  a  reliability  to  withstand 
the  vibration  treatments  at  least  97.9%  at  the 
50%  lower  confidence  level.  With  90  Items  and 
no  failure,  the  decon  1  packet  demonstrated  a 
reliability  to  withstand  the  vibration  treatment 
of  at  least  99.2%  at  the  50%  lower  confidence 
level . 


of  this  phase  of  the  evaluation  was  to:  (1) 
demonstrate  the  compatibility  of  the  nonwoven 
fabric  with  the  chemicals  of  decon  solution  1 
and  with  chloramine  B;  (2)  monitor  changes  In 
the  composition  of  decon  solution  1  and  the  loss  of 
active  chloride  from  the  chloramine  B  impregnated 
on  the  nonwoven  fabric.  The  accelerated  storage 
stability  tests  were  carried  out  at  60°C  and 
43°C,  respectively,  for  decon  1  and  decon  2  packets. 

Fabric  Properties.  Tensile  tests  were 
conducted  on  both  decon  1  and  decon  2  swatches 
to  determine  if  the  long-term  exposure  to  the 
decontaminating  agents  would  result  In  changes 
in  the  fabric  properties.  Since  the  fabric  strength 
varies  by  approximately  a  factor  of  10  depending 
on  fabric  orientation,  the  unaged  swatches  were 
tested  In  both  directions.  Ten  unaged  samples 
were  tested  in  each  direction  on  an  Instron  tensile 
tester  using  ASTM  DM  7  Section  7.1.2  procedures. 
Eight  samples  In  each  direction  were  tested  on 
the  three-month  and  the  six-month  accelerated 
storage  swatches.  Due  to  the  closeness  of  the 
average  tensile  values  In  each  of  the  four  tests, 
a  statistical  analysis  was  conducted.  Using 
t-dlstrlbutlon  statistics  to  estimate  the  true 
averages  (with  a  95%  confidence  level)  the  swatch 
tensile  strength  of  the  three-month  and  six-month 
tests  are  compared  to  the  unaged  swatches  In 
Table  2. 

Shelf-Life  Stability  of  Decon  1  Solution. 
Analyses  were  run  on  the  contents  of  packets 
kept  at  room  temperature  (control)  packets  exposed 
for  three  months  and  six  months  to  60'C,  and 
on  a  sample  taken  from  a  polypropylene  storage 
bottle  of  solution  1  .  Six  samples  were  analyzed 
of  the  accelerated  aging  tests  and  three  each 


Analysis  of  the  sample  bottle  of  Decon 
1  solution  shows  average  ammonia  and  sodium  hydrox¬ 
ide  contents  of  0.084  and  5.07  percent,  respectively. 
Sodium  hydroxide  level  Is  well  within  tolerance 
but  the  ammonia  Is  quite  low.  It  appears  that 
somewhere  In  the  process  of  formulating  and  storing 
the  solution,  ammonia  is  lost.  It  Is  also  noted 
In  this  analysis  that  in  packaging  of  the  decon 
1  solution,  approximately  20%  of  both  the  sodium 
hydroxide  and  the  ammonia  Is  lost.  This  suggests 
the  possibility  of  neutralization  by  an  acid 
constituent  either  on  the  nonwoven  fabric  or 
on  the  Inner  surface  of  the  laminates. 


Statistical  procedures  were  used  to 
determine  If  there  existed  a  significant  difference 
between  the  aged  and  the  control  samples  for 
each  constituent.  These  results  are  summarized 
in  Table  4.  The  following  observations  can  be 
made : 

(1)  No  significant  difference  exists 
in  the  phenol  contents  of  the 
control  and  aged  samples. 

(2)  The  small  loss  of  ethanol  after 
aging  three  months  Is  not  signifi¬ 
cant,  but  the  loss  after  six  months 
may  be  of  concern. 

(3)  Sodium  hydroxide  and  ammonia  concen¬ 
trations  change  appreciably  over 
time. 

Stability  of  Decon  2  Solution,  Active 
Chlorine  Content.  To  analyze  the  stability  of 
the  free  chlorine  content,  swatches  were  analyzed. 
Accelerated  storage  (43°C)  at  three  months  and 
six  months  were  compared  with  a  control  (three 
months  at  room  temperature).  Results  show  a 
very  narrow  distribution  of  data  points  with 
small  standard  deviations;  consequently,  a  t- 
test  of  the  data  was  deemed  unnecessary.  The 
results  are  summarized  in  Table  5. 

These  data  show  an  average  drop  in 
free  chlorine  content  of  4%  over  the  three-month 
storage  period  and  of  6%  over  a  six-month  period 
at  43°C.  These  results  appear  favorable  for 
the  long  range  storage  stability  of  chloramine 
B  in  the  decon  2  packets. 


Table  2.  Shelf-Life  Stability  of  Fabric 


%  Change  In 

strength 

Decon  1  Swatch 

Decon  2  Swatch 

Three-month  Test 

Strong  Direction 

0 

0 

Weak  Direction 

0 

+17 

Six-month  Test 

Strong  Direction 

0 

-14 

Weak  Direction 

+19 

10 
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Table  3.  Decon  1  Solution  Analytes 


Phenol 

- m - 

- UaUff - 

Ethanol 

Desired  Concentration  and 
Tolerance,  % 

10  +  0.5 

0.2  +  0.02 

5  +  0.5 

72  +  2.0 

Bottle,  Average  (Std  Dev) 

10.13  (0.31) 

0.084  (0) 

5.07  (0.02) 

71.71  (2.18) 

Control  (3  months,  RT) 

Average  (Std  Dev) 

9.02  (0.98) 

0.067  (0.005) 

4.07  (0.09) 

71.22  (0) 

Aged  (3  months,  60*0  Average 
(Std  Dev) 

9.31  (1.68) 

0.051  (0.0045) 

3.74  (0.02) 

70.81  (1.87) 

Aged  (6  months,  60*0  Average 
iSMJ.eyJ - 

9.44  (0.12) 

0.055  (0.0061) 

3.61  (0.04) 

66.65  (2.06) 

Table  4.  Statistical  Comparison  of  Decon  1  Packet  Contents  of 
Aged  Packets  to  Room  Temperature  Controls 


A.  3  Months  0  60°C 


Component 

Degrees  of 
freedom 

Calculated 
t-  value 

t-value  from 
table  (95% 
confidence  level) 

Significant 

Phenol 

6 

0.268 

1.943 

no 

Ethanol 

7 

1.288 

1.895 

no 

NaOH 

6 

9.851 

1.943 

yes 

nh3 

7 

4.822 

1.895 

yes 

B.  6  Months  9  60°C 


Component 

Degrees  of 
freedom 

Calculated 
t-val ue 

t-value  from 
table  (95% 
confidence  level) 

Significant 

Phenol 

7 

0.727 

1.895 

no 

Ethanol 

7 

6.14 

1.895 

yes 

NaOH 

6 

6.743 

1.943 

yes 

nh3 

7 

3.184 

1.895 

yes 

Table  5.  Active  Chlorine  Content  Versus  Time 


Average 

Standard  Deviation 

Control 

14.68 

0.116 

Accelerated  Storage 

3  months 

14.07 

0.116 

6  months 

12.77 

0.231 
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Pouch  Stability  Tests.  In  addition 
to  the  teat*  being  conducted  by  Battelle,  Mine 
Safety  Appllancea  Company  performed  analyaes 
of  the  composition  of  stored  materials.  Firstly, 

17  grams  of  solution  1  were  sealed  In  a  packet, 
minus  the  wipe,  and  stored.  After  34  months 
at  an  elevated  temperature  of  60*C  the  total 
weight  loss  was  0.10  grams.  This  equates  to 
a  loss  of  0.58X.  Also,  complete  decon  1  packets 
(with  towelette)  were  placed  In  their  cases  and 
stored.  After  nine  months  at  43'C,  0.01  grams 
of  solution  was  lost,  equating  to  a  0.058X  total 
weight  loss.  The  estimated  shelf-life  for  the 
decon  1  packet  Is  thus  estimated  at  12  years. 

The  stability  data  for  the  decon  2 
packet  are  summarized  In  Table  6.  Even  though 
there  Is  only  83X  of  the  Initial  chloramine  activ¬ 
ity  remaining  after  19  months  of  storage  at  43°C, 
the  material  Is  fully  effective.  Actually,  the 
major  loss  of  activity  occurs  initially  and  then 
tapers  off.  The  estimated  shelf-life  Is  eight 
years . 

HEL  Testing 

Objective.  CSL  went  to  the  U.  S.  Army  Human 
Engineering  Laboratory  (HEL),  Aberdeen  Proving 
Ground,  Maryland,  to  determine  If  the  user  could 
operate  the  product  Improved  version  of  the  M2 5 8 
Skin  Decontamination  Kit  as  well  as  or  better 
than  the  existing  M258  Kit.  Additionally,  the 
HEL  was  to  determine  If  the  user  could  distinguish 
between  the  decon  solutions  1  and  2  envelopes 


In  low-level  light  condition  and  to  determine 
If  the  user  could  succeesfully  manipulate  the 
decon  1  and  2  envelopes  while  wearing  chemical 
protective  gloves.  These  tests  used  the  MSS 
and  the  PIP  M58  training  kits. 

Test  Conditions.  This  test  was  designed 
to  evaluate  the  differences  in  performance  times 
and  ease  of  preparing  the  pads  with  eolutlons 
I  and  II.  Test  participants  consisted  of  ten 
U.  S.  Army  personnel  trained  In  the  use  of  each 
kit.  Each  kit  was  evaluated  and  compared  under 
three  test  conditions:  (1)  bare  handed,  (2) 
wearing  chemical  protective  gloves,  and  (3)  wearing 
chemical  protective  gloves  and  blacked-out  goggles 
to  simulate  night  conditions.  Performance  times 
were  measured  for  conditions  1  and  2.  However, 
for  condition  3,  only  the  observations  and  comments 
of  test  personnel  were  used  for  evaluative  purposes. 

In  order  to  balance  the  learning  effect 
that  results  from  repeated  training  and  testing 
trials,  the  ten  test  participants  were  divided 
Into  two  groups  of  five  each.  Group  I  was  tested 
first  with  the  standard  M58  training  kit  and 
second  with  the  PIP  training  kit.  The  testing 
order  of  the  kits  was  then  reversed  for  Group 
II. 

Result^.  Median  performance  times  for  condi¬ 
tions  1  and  2  are  shown  In  Table  7.  The  only  steps 
that  are  different  for  the  standard  M58  kit  and 
the  PIP  kit  are  the  preparations  of  the  decon 
pads  with  solutions  I  and  II  or  1  and  2.  As 


Table  6.  Decon  2  Pouch-Chloramine  B  Stability 


%  Initial  Activity' 

Chloramine  B  Crystals 

24  Months  at  43“C  In  sealed  glass  ampoule 

99.6 

24  Months  at  43°C  In  sealed  foil  pouch 

99.1 

Chloramine  B  Impregnated  on  Rayon/Polypropylene  Fabric 

10  Months  at  438C 

86 

19  Months  at  43°C 

83 

TaDle  7.  Median  Performance  Times*  Comparing  the 
M58  Decon  Kit  and  the  PIP  Version 


Task 

Barehanded 

61 oved 

PIP 

M58 

Mask  time 

16 

14 

Open  kit 

4 

5 

3 

3 

Wipe  with  dry  pad 

31 

— 

Prepare  pad  with  Solution  I  (1) 

41 

11 

44 

14 

Wipe  with  pad  1 

69 

80 

Prepare  pad  with  Solution  II  (2) 

46 

21 

45 

24 

Wipe  with  pad  2 

96 

118 

Decon  face 

49 

32 

Don  gloves 

23 

28 

TOTAL 

391 

330 

98 

43 

*  Time  measured  in  seconds 


571 


ABOUT  THE  AUTHOR 


can  be  eeen  from  the  table,  the  preparation  tine 
of  the  decon  1  pad  la  ahortened  by  approxlaately 
75%  with  the  PIP  kit.  Preparation  tlae  of  the  Mr.  Jaaea  M.  Saachta,  Project  Manager 

decon  2  pad  ia  reduced  by  more  than  50%  with  Produclblllty  Engineering 

the  PIP  kit.  Branch/Phyalcal  Protection  Divialon 


Preparation  of  a  Technical  Data  Package  (TOP) 


At  the  conclualon  of  the  PIP  effort,  dravinge 
and  specif lcatlona  were  prepared  that  could  be 
used  for  competitive  procurement  act  Iona.  Thla 
TOP  waa  completed  and  forwarded  to  ARRCOM  and 
has  subsequently  been  uaed  in  awarding  a  Government 
procurement  contract. 


SUMMARY 

A  one-year  Joint  effort  (in  FY78)  between 
CSL  and  Mine  Safety  Appliances  Company  resulted 
in  an  Improved  version  of  the  M258  Personal  Skin 
Decontamination  Kit.  In  FY79,  a  product  Improvement 
program  was  launched,  and  completed  in  approximately 
14  months.  The  PIP  effort  included  finalisation 
of  kit  design,  biomedical  testing,  environmental 
testing,  and  human  engineering  testing.  Finally, 
a  technical  data  package  was  prepared.  Medical 
approval  was  granted  by  the  Surgeon  General  in 
July  1980  for  the  M258/M58  PIP  materials. 


Chemical  engineer.  Develops,  supervises,  manages 
and  coordinates  product  Improvement  programs; 
manufacturing  methods  technology  programs;  and 
engineering  support  efforts  for  chemical  decontam¬ 
ination  materials,  procedures,  and  equipment. 

Chemical  Systems  Laboratory 
U.  S.  Army  ARRADCOM 
Aberdeen  Proving  Ground,  Maryland 


The  product  improved  kits  obviated  several 
safety  hazards  potentially  available  In  the  original 
M258  kits.  They  contain  less  of  the  active  ingredi¬ 
ents  (per  packet)  while  still  providing  adequate 
amounts  lor  effective  decontamination. 

The  potential  for  eye  irritation  is  consider¬ 
ably  lessened  by  the  fact  that  in  towelette  form, 
without  excessive  fluid,  the  possibility  of  splash¬ 
ing  liquid  into  the  eye  seems  remote.  When  this 
is  coupled  with  user  instructions  and  training 
to  avoid  getting  the  materials  into  the  eye, 
and  that  decontamination  should  only  be  conducted 
on  the  lower  face,  the  hazard  appears  quite  limited 
and  acceptable,  particularly  considering  the  benefit 
to  be  gained  by  having  an  effective  decontamination 
process  for  lethal  CW  agents. 

The  product  improved  kit  was  tested  by 
Army  personnel  (officer  and  enlisted)  In  training 
exercises.  Afterwards,  the  personnel  were  asked 
to  complete  a  questionnaire.  All  participants 
felt  that  the  product  improved  kit  was  a  marked 
improvement  over  the  standard  M58  kit  in  all 
steps  of  operation. 


Each  kit  contains  three  complete  decontamina¬ 
tions,  and  it  has  been  demonstrated  that  the  product 
improved  kit  has  operational  equivalence  to  the 
M258  Kit  plus  also  providing  a  facial  decon. 

Indeed,  the  PIP  kit,  because  of  the  pretreated 
towelettes  and  tear-away  packet  design,  has  Bhown 
enhanced  effectiveness  in  neutralizing  agent 
since  it  is  easier  and  faster  to  use.  It  has 
also  been  demonstrated  that  the  PIP  kit  can  be 
used  to  decontaminate  small  areas  of  personal 
equipment  such  as  rifle,  gloves,  etc.  Finally, 
costs  have  been  reduced  substantially.  The  PIP  kit 
can  be  fabricated  and  assembled  for  approximately 
the  same  cost  as  the  old  M258  kit,  yet  it  contains 
three  complete  decontaminations  as  opposed  to 
one.  The  PIP  kit  and  the  PIP  training  kit  have 
been  designated  the  M258A1  and  M58A1 ,  respectively. 
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